Use Spring to handle x www form urlencoded mode

Directory Spring processing x-www-form-urlencoded mode on application/x-www-form-urlencoded encoding

Spring processing x-www-form-urlencoded mode

Recently, I encountered many wonderful problems when rewriting a project. This project is a simple web background project, which is basically all the operations of adding, deleting, modifying and checking the database. There are several interfaces that use spring to receive front-end post requests.

The basics are that post requests have four data parameter formats, the basics of which I mentioned in another blog post, so don't talk nonsense here. The main reason is that this interface is used in two places at the front end, but when using this interface, the usage of the two places is different, and the wonderful c + + has been successfully parsed (in fact, because c + + did not check the request parameter format and data, there is no problem).

One place is sent in application/json format, and one jsonArray data is sent (data example ["abc", "bcd"]). This is the correct way to use it without problems. (Hereinafter referred to as the former)

The other place is in the format of application/x-www-form-urlencode, and also sends one jsonArray data. (Hereinafter referred to as the latter)

The former is simple in analysis

 @RequestMapping(value = "/check_apps_version",
      method = RequestMethod.POST,
      produces = {"application/json;charset=UTF-8"},
      consumes = MediaType.APPLICATION_JSON_UTF8_VALUE)
  @ResponseBody
  public BaseResponse<List<AppListItem>> checkAppsVersion(ReqCheckAppsVersion requstParam,
      @RequestBody List<String> apps) {
    return new BaseResponse<>();
  }

The latter sending mode is very strange when spring is parsed with @ RequestBody, but the previous stage is that the mobile phone APP has been released and cannot be modified, so it can only be modified at the back end to meet this strange demand

Through debugging, it is found that the parameters passed from the interface of the front end of the latter are "[" abc "," def "] =" In this way, x-www-form-urlencode is the data format of multiple kev-value pairs, so there is no value but key now, which can only be solved by string processing.

  @RequestMapping(value = "/check_apps_version",
      method = RequestMethod.POST,
      produces = {"application/json;charset=UTF-8"},
      consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
  @ResponseBody
  public BaseResponse<List<AppListItem>> checkAppsVersionParams(HttpServletRequest request,
                                                                ReqCheckAppsVersion requstParam,
                                                                @RequestBody String apps) {
 String body = request.getReader().lines().collect(Collectors.joining(System.lineSeparator()));
    return BaseResponse.success();
  }

There are two ways, one is to parse post data into string format through @ RequestBody, and the other is to parse the whole original post data through HttpServletRequest. Then do string processing.

But when we did this test, we thought about whether there would be only value without key, that is, "= [" abc "," def "]".

The test result is that HttpServletRequest can't be parsed from HttpServletRequest to post data with tomcat, but jetty can be parsed from HttpServletRequest to post data.

This may be the difference between tomcat and jetty, but it is not clear why. But our problem has finally been solved, and the biggest feeling is that our predecessors dug holes and later generations buried them.

I hope I can pay attention to the problem of code robustness in the future and avoid digging holes for others or myself.

On application/x-www-form-urlencoded coding

Colleagues encountered the problem that the data from POST was read through request object getInputStream at servlet, but could not be read, which was suspected to be the problem of tomcat. After checking that Content-type is application/x-www-form-urlencoded, it is estimated that it was resolved into parameters. Indeed, before he obtained the stream, he had the operation of request. getParameter.

If you are familiar with servlet, this problem should be considered common sense. It has nothing to do with containers, and all servlet containers behave like this. A few years ago, when implementing a gateway proxy, I encountered this problem. At that time, I used jetty, and found that the data from POST could not be read. It was also encoded by application/x-www-form-urlencoded. Breakpoint tracking found that there was request. getParameter before obtaining the stream, and the data would be parsed and the subsequent data stream could not be read again.

In section 3.1. 1 of the servlet specification, there is a description of when POST data will be treated as parameters:

1. The request is an HTTP or HTTPS request.
2. The HTTP method is POST.
3. The content type is application/x-www-form-urlencoded.
4. The servlet has made an initial call of any of the getParameter family of methods on the request object.

If the conditions are met, post form data will no longer be available for reading directly from the request object's input stream.

It has been explicitly stated in the specification that when the request is satisfied:

1) http/https

2) POST

3) Content-type is application/x-www-form-urlencoded

4) The getParameter method has been called, the data is treated as the requested paramaters and can no longer be read directly through the inputstream of the request.

Therefore, tomcat, jetty, and other servlet containers follow this approach. But then again, why is the data encoded by application/x-www-form-urlencoded parsed as parameter?

Upload data with http can use GET or POST. If you use GET, you can only use queryString of uri, which will encounter the problem of length. Different browsers or server may support different lengths, so if the data to be submitted is too long, it is not suitable to submit with GET.

With POST, you can either have queryString in uri or put data in body. The content of body can be encoded in many forms, among which application/x-www-form-urlencoded encoding is actually based on percent-encoding encoding of uri, so POST data using application/x-www-form-urlencoded and queryString are only different in form and essentially transfer parameters.

In the Request. parseParameters method of tomcat, there is a judgment on application/x-www-form-urlencoded. For this coding, the data in body will be parsed and filled into parameters, so it is impossible to read body by stream later (unless you have not triggered getParameter related methods).

Before HTML4, the encoding method of form data was only application/x-www-form-urlencoded (which is also the default method now), because in the early days, the data submitted on web was also very simple, basically in the form of key-value, so the form adopted application/x-www-form-urlencoded

multipart/form-data code is introduced in HTML4. Please refer to here for how to choose these two codes.