Solve the problem that the percent sign contained in RestTemplate request url will be escaped to 25

Directory RestTemplate Request url contains a percent sign that will be escaped to 25 Solution RestTemplate Transcoding bug Transcoding Problem Background Conclusion Why is this problem?

The percent sign contained in the RestTemplate request url is escaped to 25

Initially using RestTemplate to make remote calls is as follows:

private String getRemoteData(String url) {
  logger.info("Request URL :" + url + "|");
 
  String resp = rest.getForObject(url, String.class);
 
  logger.info("Response result : " + resp.toString());
  return resp;
 }

However, it is found that the request result 1 is straight empty.

Finally, it is found that in our business scenario, the request parameters contain Chinese requirements and transcode according to the specified rules, which leads to the request url containing%, while RestTemplate will automatically call encode method to escape, escaping% to% 25.

Solution

Self-built URI incoming:

private String getRemoteData(String url) {
  logger.info("Request URL :" + url + "|");
  String resp = null;
  try {
   URI uri = new URI(url);
   resp = rest.getForObject(uri, String.class);
  } catch (URISyntaxException e) {
   logger.error("Create URI Exception !");
  }
 
  logger.info("Response result : " + resp.toString());
  return resp;
 }

RestTemplate Transcoding bug

A rare bug request for Get was found for HTTP.

Background of transcoding problem

It is necessary to send a complex get request to the tigergraph server, with only one parameter, but the value of the parameter is a complex json

The value received by the server is always abnormal. The observation shows that the anomaly is that everything that should have been resolved as spaces on the server side becomes a plus sign (+).

Think there is something wrong with the code, and then use the native way of HTTPclient to initiate the request:

public static String doGet(String url) throws Exception{
        HttpGet get = new HttpGet(url);
        return doMethod(get);
    }

    private static String doMethod(HttpRequestBase method)throws Exception{
        CloseableHttpResponse response = null;
        CloseableHttpClient client;
        HttpClientBuilder hcb = HttpClientBuilder.create();
        HttpRequestRetryHandler hrrh = new DefaultHttpRequestRetryHandler();
        HttpClientBuilder httpClientBuilder = hcb.setRetryHandler(hrrh);
        client = httpClientBuilder.build();
        method.addHeader("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;SV1)");
        method.addHeader(HTTP.CONTENT_TYPE, APPLICATION_JSON);
        RequestConfig.Builder confBuilder = RequestConfig.custom();
        confBuilder.setConnectTimeout(CONNECT_TIMEOUT);
        confBuilder.setConnectionRequestTimeout(REQUEST_TIMEOUT);
        confBuilder.setSocketTimeout(SOCKET_TIMEOUT);
        RequestConfig config = confBuilder.build();
        method.setConfig(config);
        response = client.execute(method);
        int code = response.getStatusLine().getStatusCode();
        String result = EntityUtils.toString(response.getEntity());
        response.close();
        client.close();
        return result;
    }

Getting results is still a problem, as is building http requests using the Assured test tool.

Conclusion

Later, after carefully checking the URLEncode. encode method and RestTemplate source code implementation, it is found that the transcoding protocol of the client and the decoding protocol of the server do not match.

After repeated tests and serious, this problem only occurs when there is a space in the parameter, and no other characters, such as:/* & None of these special characters have this problem.

The final solution is to replace the URL string with% 20 spaces in the transcoded string, and then use http client native request mode.

The second solution is to use the UriComponentsBuilder class of RestTemplate, use (builder. build (false). toUri () to get URL, and the parameter must be false to turn the space to% 20

/** * urlencode Transcoding can't be used casually, because it will convert spaces into + No., not standard %20 Characters.  *  For spring The server side built will not have this problem. But I'm in tiger This problem was encountered on the server.  *  So urlencode The protocols that apply only to server-side support are RFC1738 *  If the server only supports RFC 2396 Standard, when the server decodes, it will put the plus sign + As reserved characters, but not transcoded  * */
  @Override
    @SuppressWarnings("all")
    public <Req, Resp> Resp doGet(String url, Req request, Class<Resp> responseType) throws Exception {
        UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url);
        Map<String, Object> parameters = (Map<String, Object>)request;
        for (Map.Entry<String, Object> entry : parameters.entrySet()) {
            builder.queryParam(entry.getKey(), Objects.toString(entry.getValue(), ""));
        }
        return restTemplate.getForObject(builder.build(false).toUri(), responseType);
    }

Why is there such a problem?

The URLEncode class rooted in Java language can only be applied to the early RFC protocol, and usually the server developed by spring is compatible with this pattern.

The new version of the RFC protocol will treat the + sign as a keyword and no longer reverse it into a space, which is usually reflected in new technologies, such as the current tigergraph graph database.