Detailed Explanation of Node Data Verification Process in Controller Layer

Preface

Humorous back-end programmers laugh at themselves as CURD Boy. CURD, that is, the addition, deletion and modification of a certain storage resource, which is completely data-oriented programming.

Good, data-oriented programming tends to have a better understanding of the business, resulting in higher quality code and fewer BUG. Since it is data-oriented programming, it is more necessary to avoid the appearance of dirty data and strengthen data verification. Otherwise, do you want to believe in front-end data verification? After all, front-end data verification goes directly to users for more friendly user feedback in UI layer.

Data verification layer

The back-end is divided into various levels because of its emphasis on business logic and various data to be processed. The back-end projects I have experienced are divided into various named layers such as Controller, Service, Model, Helper and Entity, with 5 flowers and 8 doors. But there must be a layer called Controller, which stands at the top of the back end and directly receives the data transmitted by the client.

Because Controller layer is the top level of data interaction between server and client, adhering to the principle of Fail Fast, it shoulders the function of data filter, and directly calls back illegal data, which is as dignified as Qin Qiong and Weichi Gong.

At the same time, data verification derives a semi-documented by-product. You only need to look at the data verification layer to know which fields to pass and what format they are.

The following are common data validation, and this article shows how to validate them:

required/optional Basic data verification, such as number, string, timestamp and the conditions that the value needs to meet Complex data verification, such as IP, mobile phone number, email address and domain name

const body = {
 id,
 name,
 mobilePhone,
 email
}

Shanyue came into contact with a back-end project without data verification layer, and if/else was full of various levels, which was extremely painful and reconstructed in minutes.

JSON Schema

JSON Schema is based on JSON for data verification format, with a specification json-schema. org. At present (2020-08), the latest version is 7.0. Various server programming languages have implemented the specification, such as go, java, php, etc. Of course, there are also great javascript, such as tepid ajv.

The following is an Schema for verifying user information, which shows that the syntax is complicated and cumbersome:

{
 "$schema": "http://json-schema.org/draft-04/schema#",
 "title": "User",
 "description": " User information ",
 "type": "object",
 "properties": {
 "id": {
 "description": " Users  ID",
 "type": "integer"
 },
 "name": {
 "description": " User name ",
 "type": "string"
 },
 "email": {
 "description": " User mailbox ",
 "type": "string",
 "format": "email",
 "maxLength": 20
 },
 "mobilePhone": {
 "description": " User mobile phone number ",
 "type": "string",
 "pattern": "^(?:(?:\+|00)86)?1[3-9]\d{9}$",
 "maxLength": 15
 }
 },
 "required": ["id", "name"]
}

For complex data type verification, JSON Schema has the following built-in Format, which is convenient and quick to verify

Dates and times Email addresses Hostnames IP Addresses Resource identifiers URI template JSON Pointer Regular Expressions

For mobile phone numbers that are not in the built-in Format, use ajv. addFormat to add Format manually

ajv.addFormat('mobilePhone', (str) => /^(?:(?:\+|00)86)?1[3-9]\d{9}$/.test(str));

Joi

joi claimed to be the most powerful JS verification library, and also won 16,000 stars in github. Compared with JSON Schema, its syntax is more concise and powerful.

The most powerful data validation library for JS

Completing the same validation requires less code and can complete more powerful validation. The following are examples only. Please go to the documentation for more examples.

const schema = Joi.object({
 id: Joi.number().required(),
 name: Joi.number().required(),
 email: Joi.string().email({ minDomainSegments: 2, tlds: { allow: ['com', 'net'] } }),
 mobilePhone: Joi.string().pattern(/^(?:(?:\+|00)86)?1[3-9]\d{9}$/),

 password: Joi.string().pattern(/^[a-zA-Z0-9]{3,30}$/),
 //  And  password  Same check 
 repeatPassword: Joi.ref('password'),
})
 //  Password and duplicate password need to be sent at the same time 
 .with('password', 'repeat_password');
 //  Email box and mobile phone number are provided 1 One is enough 
 .xor('email', 'mobilePhone')

Data verification and routing layer integration

Because the data is directly transmitted from the route, koajs officially implements an joi-router based on joi, and the pre-data is verified to the routing layer to verify query, body and params transmitted from the front end.

joi-router also analyzes and limits various content-type transmitted at the front end based on co-body. If restricted to application/json, CSRF attack can also be prevented to a certain extent.

const router = require('koa-joi-router');
const public = router();

public.route({
 method: 'post',
 path: '/signup',
 validate: {
 header: joiObject,
 query: joiObject,
 params: joiObject,
 body: joiObject,
 maxBody: '64kb',
 output: { '400-600': { body: joiObject } },
 type: 'json',
 failure: 400,
 continueOnError: false
 },
 pre: async (ctx, next) => {
 await checkAuth(ctx);
 return next();
 },
 handler: async (ctx) => {
 await createUser(ctx.request.body);
 ctx.status = 201;
 },
});

Regular Expressions and Safe Regular Expressions

Shan Yue found that an API took too long in the data verification layer when troubleshooting performance problems, which I never thought of. The root of the problem lies in unsafe regular expressions, so what is an unsafe regular expression?

For example, the following regular expression that can hang CPU is a time bomb, and the number of backtracks has increased exponentially.

You can refer to the article to analyze the principle and practice of ReDos

const safe = require('safe-regex')
const re = /(x+x+)+y/

//  Can run to death  CPU  Adj. 1 A regular 
re.test('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

//  Use  safe-regex  Determine whether regularity is safe or not 
safe(re) // false

Data validation is mostly aimed at string validation, and it is also full of various regular expressions, so it is very important to ensure the safety of regular expressions. safe-regex can find out which unsafe regular expressions.

Summarize

The Controller layer needs to check the data of system 1, and JSON Schema (Node realizes ajv) and Joi can be used JSON Schema has the official specification and the realization of each language, but the syntax is cumbersome, and Joi with more powerful verification function can be used Pay attention to the performance problems caused by unsafe regularization when performing string validation