Net Core HttpClient Processing Response Compression Details

  • 2021-12-04 18:27:25
  • OfStack

Directory 1. How to use it 2. Source code exploration

Foreword:

As mentioned in the previous article ASP. NET Core implementation of response compression, the main work of the server side is based on Content-Encoding The header information determines which way to compress and return. Someone in the group asked before, is it necessary to compress requests on the server side when the network bandwidth is so high? Indeed, nowadays, distributed and load balancing technologies are so mature that many scenarios that need to deal with highly concurrent big data can be carried out by adding server nodes. However, under the condition of limited resources, or when it is not necessary to add new server nodes for a certain point, we still need to use some conventional processing methods of the program itself to deal with it. Personally, the author thinks that the use scenario of response compression is like this. When the bandwidth pressure is tight and CPU resources are sufficient, the overall effect of using response compression is obvious.
Where there is compression, there is decompression, and the decompression work is handled at the requesting client. For example, browsers, which are our most commonly used Http clients, many browsers default when we make requests (such as when we browse the web) in Request Head Add to Content-Encoding And then process the related decompression based on the response information. These are due to the fact that the browser has built-in mechanisms for request compression and decompression. There are many similar ones, such as the commonly used agent package grabbing tool Filder This mechanism is also built in. It just needs to be handled manually, but the implementation is all the same. Sometimes we need to use this mechanism in the process of writing our own programs. In the traditional .Net HttpWebRequest Class library, there is no such mechanism, and later versions added HttpClient Has its own mechanism to handle this operation,

1. How to use it

First of all, let's look at 1 directly in HttpClient How to handle response compression in


// Customize HttpClientHandler Instances 
HttpClientHandler httpClientHandler = new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.GZip
};
// Use pass customization HttpClientHandler Constructor of an instance 
using (HttpClient client = new HttpClient(httpClientHandler))
{
    var response = await client.GetAsync($"http://MyDemo/Home/GetPerson?userId={userId}");
}


This operation is still very simple, and what we are operating is not HttpClient The attribute of is HttpClientHandler Attributes in our previous article [ .NET Core HttpClient Source code exploration], HttpClient The essence of is actually Request Head1 , and HttpClient What is really used is Request Head1 The most important 1 subclass HttpClientHandler All request operations are performed through the Request Head1 In progress. We can see that Request Head6 What is accepted is that Request Head7 Enumeration, since it is an enumeration, it means that it contains more than 1 value. Next, let's look at Request Head7 Source code in


[Flags]
public enum DecompressionMethods
{
    //  Use all compression and decompression algorithms. 
    All = -1,
    //  Do not use decompression 
    None = 0x0,
    //  Use gzip Decompression algorithm 
    GZip = 0x1,
    //  Use deflate Decompression algorithm 
    Deflate = 0x2,
    //  Use Brotli Decompression algorithm 
    Brotli = 0x4
}


This enumeration is aimed at common output decompression algorithms by default. Next, let's look at 1. Request Head9 Gets or sets how to handle response compression in. In the previous article [. NET Content-Encoding0 Implement service discovery], which we discussed Request Head9 The approximate way of working of defaults Content-Encoding2 The HttpClientHandler instance is passed, and the HttpClientHandler instance is passed when we register Request Head9 When it is possible to pass Content-Encoding4 Customize Content-Encoding2 The default value of, then we will implement the specific code


services.AddHttpClient("mydemo", c =>
{
    c.BaseAddress = new Uri("http://MyDemo/");
}).ConfigurePrimaryHttpMessageHandler(provider=> new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.GZip
});


Actually registering Request Head9 You can also use the custom HttpClient The specific way of using it is as follows


services.AddHttpClient("mydemo", c =>
{
    c.BaseAddress = new Uri("http://MyDemo/");
}).ConfigureHttpClient(provider => new HttpClient(new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.GZip
}));


HttpClient Really helped us to do a lot of things, only need a simple configuration under 1 to start the response compression processing. This reminds us of our interest in HttpClient Next, we will look at how it initiates a responsive compression request and decompresses the response result through the source code.

2. Source code exploration

Through the above usage, we know that no matter which form is used, it is ultimately aimed at HttpClientHandler Do the configuration operation, and then let's look at the HttpClientHandler Class Request Head6 The code of the property


public DecompressionMethods AutomaticDecompression
{
    get => _underlyingHandler.AutomaticDecompression;
    set => _underlyingHandler.AutomaticDecompression = value;
}

Its own value operation comes from _underlyingHandler This object, that is to say, reading and setting are both operations _underlyingHandler.AutomaticDecompression , we found _underlyingHandler The declared location of the object


private readonly SocketsHttpHandler _underlyingHandler;

Illustrated here under 1, HttpClient The substantive work class of is HttpClientHandler , and HttpClientHandler Really initiating a request depends on SocketsHttpHandler This class, which means SocketsHttpHandler Is the class that originally initiated the request. HttpClientHandler Essence or through SocketsHttpHandler The Http request initiated, and then we will look at the SocketsHttpHandler Class handles Request Head6 Object of this property


public DecompressionMethods AutomaticDecompression
{
    get => _settings._automaticDecompression;
    set
    {
        CheckDisposedOrStarted();
        _settings._automaticDecompression = value;
    }
}

Here's _settings Is no longer a concrete functional class, but is used to initialize or save SocketsHttpHandler Configuration class for some property values of


private readonly HttpConnectionSettings _settings = new HttpConnectionSettings();

We are not analyzing here SocketsHttpHandler Out to deal with other code except response compression, so we will no longer look at these specifically and find them directly _settings._automaticDecompression Property, and finally found this code


if (settings._automaticDecompression != DecompressionMethods.None)
{
    handler = new DecompressionHandler(settings._automaticDecompression, handler);
}

It is clear here that the real processing of request response compression is related to the DecompressionHandler Medium. As we said before, HttpClient The real way to work is to realize something from Request Head1 The subclass of, which encapsulates the implementation modules of different functions into concrete HttpClient2 Medium. When you need to use the function of which module, directly use the corresponding HttpClient2 Operation class to send processing requests. This kind of design idea lies in HttpClient4 Is also reflected in incisively and vividly, HttpClient4 It is used to build different endpoints to process and output requests. From these, we can know that DecompressionHandler Is the topic today, so let's look at it next DecompressionHandler Class source code, let's first look at the core HttpClient8 Method, which is the execution method that sends the request


internal override async ValueTask<HttpResponseMessage> SendAsync(HttpRequestMessage request, bool async, CancellationToken cancellationToken)
{
    // Determine whether it is GZIP Compress the request and add the request header if it is Accept-Encoding Head is gzip
    if (GZipEnabled && !request.Headers.AcceptEncoding.Contains(s_gzipHeaderValue))
    {
        request.Headers.AcceptEncoding.Add(s_gzipHeaderValue);
    }
    // Determine whether it is Deflate Compress the request and add the request header if it is Accept-Encoding Head is deflate
    if (DeflateEnabled && !request.Headers.AcceptEncoding.Contains(s_deflateHeaderValue))
    {
        request.Headers.AcceptEncoding.Add(s_deflateHeaderValue);
    }
    // Determine whether it is Brotli Compress the request and add the request header if it is Accept-Encoding Head is brotli
    if (BrotliEnabled && !request.Headers.AcceptEncoding.Contains(s_brotliHeaderValue))
    {
        request.Headers.AcceptEncoding.Add(s_brotliHeaderValue);
    }
    // Send a request 
    HttpResponseMessage response = await _innerHandler.SendAsync(request, async, cancellationToken).ConfigureAwait(false);

    Debug.Assert(response.Content != null);
    // Gets the returned Content-Encoding Output header information 
    ICollection<string> contentEncodings = response.Content.Headers.ContentEncoding;
    if (contentEncodings.Count > 0)
    {
        string? last = null;
        // Get the last 1 Values 
        foreach (string encoding in contentEncodings)
        {
            last = encoding;
        }
        // According to the response header, it is judged whether the server adopts is gzip Compression 
        if (GZipEnabled && last == Gzip)
        {
            // Use gzip The decompression algorithm decompresses the returned content and assigns it from the new value to the response.Content
            response.Content = new GZipDecompressedContent(response.Content);
        }
        // According to the response header, it is judged whether the server adopts is deflate Compression 
        else if (DeflateEnabled && last == Deflate)
        {
            // Use deflate The decompression algorithm decompresses the returned content and assigns it from the new value to the response.Content
            response.Content = new DeflateDecompressedContent(response.Content);
        }
        // According to the response header, it is judged whether the server adopts is brotli Compression 
        else if (BrotliEnabled && last == Brotli)
        {
            // Use brotli The decompression algorithm decompresses the returned content and assigns it from the new value to the response.Content
            response.Content = new BrotliDecompressedContent(response.Content);
        }
    }
    return response;
}

From the above logic, we can see that HttpClient9 , HttpClient0 , HttpClient1 Three variables of bool type, three of which determine which request compression mode is adopted, and the main implementation mode is


[Flags]
public enum DecompressionMethods
{
    //  Use all compression and decompression algorithms. 
    All = -1,
    //  Do not use decompression 
    None = 0x0,
    //  Use gzip Decompression algorithm 
    GZip = 0x1,
    //  Use deflate Decompression algorithm 
    Deflate = 0x2,
    //  Use Brotli Decompression algorithm 
    Brotli = 0x4
}


0

Mainly according to our configuration Request Head7 Enumeration value to determine which way of compression results you want to get, and the implementation logic of decompression is encapsulated in HttpClient3 , HttpClient4 , HttpClient5 Let's take a look at their specific code


[Flags]
public enum DecompressionMethods
{
    //  Use all compression and decompression algorithms. 
    All = -1,
    //  Do not use decompression 
    None = 0x0,
    //  Use gzip Decompression algorithm 
    GZip = 0x1,
    //  Use deflate Decompression algorithm 
    Deflate = 0x2,
    //  Use Brotli Decompression algorithm 
    Brotli = 0x4
}


1

Its main working mode is to use the decompression method corresponding to the compression algorithm to get the original information. To sum up briefly, HttpClient The processing mechanism related to compression is, first of all, according to your configuration Request Head7 Determine which compression algorithm you want to use. Then add Accept-after matching with the corresponding compression algorithm- HttpClient8 The request header is the compression algorithm you expect. Finally, it is obtained according to the response result Content-Encoding Output header information, judge which compression algorithm is used by the server, and use the corresponding decompression method to decompress and obtain the original data.

Summary:

Through this discussion, HttpClient As we can see about the processing of response compression, HttpClient It has high flexibility and extensibility in both design and implementation, which is why it is HttpClient2 The official only recommends making 用HttpClient 1 Http request mode. Because the use is relatively simple and the implementation method is relatively clear, it will not be described too much here. The main purpose is to tell everyone HttpClient By default, response compression can be handled directly instead of the same as that we used before HttpWebRequest It also needs to be implemented by manual coding.


Related articles: