Configuration method to get the value of the large file MD5 from the Nginx server

The HTTP protocol adds the Content-MD5 HTTP header, but nginx does not support this feature, and officials have made it clear that it will not be added. Why? Because the entire file needs to be read for each request to calculate the MD5 value, nginx, which is known for its performance, will never want to do anything that goes against the purpose of the software. However, in some applications, you need to verify the correctness of the file. Some people download the current file and then calculate the MD5 value to check whether the current file is correct or not. Not only is it a waste of bandwidth, it's a waste of time. Where there is a need, there is a solution. Netizens have developed the file-md5 module.
1. Download module file-md5

  # cd /usr/local/src
  # wget https://github.com/cfsego/file-md5/archive/master.zip -O file-md5-master.zip
  # unzip file-md5-master.zip

2. Install module file-md5

  # wget http://nginx.org/download/nginx-1.4.2.tar.gz
  # tar -xzf nginx-1.4.2.tar.gz
  # cd nginx-1.4.2
  # ./configure --prefix=/usr/local/nginx-1.4.2 --add-module=../file-md5-master
  # make
  # make isntall

If you already have nginx installed, just add the file-md5 module.

3. The configuration file - md5
3.1 MD5 is appended to the http response header

  server {
    listen    80;
    server_name test.ttlsa.com;
    root /data/site/test.ttlsa.com;
  
    # for add content-md5 to http header
    location ~ /download
    {
        add_header  Content-MD5  $file_md5;
    }
  }

For all requests to download, Content-MD5 will be added to the response http header, with the value of MD5 of this file, as shown in the following test:

  # curl -I test.ttlsa.com/download/1.exe  
  HTTP/1.1 200 OK
  Server: nginx
  Date: Wed, 26 Feb 2014 03:00:05 GMT
  Content-Type: application/octet-stream
  Content-Length: 1535488
  Last-Modified: Mon, 24 Feb 2014 10:08:10 GMT
  Connection: keep-alive
  ETag: "530b1a0a-176e00"
  Content-MD5: 6adda4a06dbad3ac9b53a08f4ff9c4f8
  Accept-Ranges: bytes

You can see Content - MD5:6 adda4a06dbad3ac9b53a08f4ff9c4f8, this is 1. exe MD5 value of files.
3.2 direct response MD5 values into the content

  server {
    listen    80;
    server_name test.ttlsa.com;
    root /data/site/test.ttlsa.com;
 
    # for add content-md5 to http header
    location ~ /download
    {
      if ( $arg_md5 ~* "true" ){
        echo $file_md5;
      }
    }
  }

Here, echo is directly used to output the MD5 value (echo module requires additional installation). Just add the parameter &md5=true to the downloaded file to get the MD5 value. During the use, the parameter can be defined at will. So let's test 1.

  # curl test.ttlsa.com/download/1.exe?md5=true  
  6adda4a06dbad3ac9b53a08f4ff9c4f8

Get the md5 value directly, and get the same MD5 as the first method.
4. The last
Using the nginx module is also a method, which has the disadvantage that each request needs to be re-calculated once to calculate the MD5 value. To reduce his pressure, you can add the cache in nginx, or borrow memcache and use perl or lua modules. I hope you can continue to support the operation and maintenance time.
Address of the project: https: / / github com/cfsego/file - md5
Project documentation: https: / / github com cfsego/file md5 / blob/master/README

PS: nginx large file download optimization
By default, proxy_max_temp_file_size value is 1024MB, that is to say, the files of the back-end server can be cached into the nginx proxy hard disk if they are not larger than 1G, if they are larger than 1G, the files will not be cached, but will be directly forwarded to the client. If proxy_max_temp_file_size is set to 0, indicating that no temporary cache is used.
In a large file environment, if you want to enable temporary caching, you can change the configuration and change the value to what you want.
Modify the nginx configuration

location /
 {
 ...
 proxy_max_temp_file_size 2048m;
 ...
 }

Restart nginx

# /usr/local/nginx-1.7.0/sbin/nginx -s reload