Nginx implements a reverse proxy instance of a static resource

  • 2020-05-09 19:54:29
  • OfStack

Many projects in github have an readme file, and many people like to add their own creation or cover image to the file. For example, substack draws an logo for each of his projects. These images are displayed directly on the page in github, but url has been replaced with github's own. For example, in the browserify project, the link to logo becomes


https://camo.githubusercontent.com/e19e230a9371a44a2eeb484b83ff4fcf8c824cf7/687474703a2f2f737562737461636b2e6e65742f696d616765732f62726f777365726966795f6c6f676f2e706e67

And we can see by looking at raw that the original url is

http://substack.net/images/browserify_logo.png

One of the benefits of doing this is to prevent you from getting a risk warning on the client side because the http link appears on the https site. github is really thoughtful about the details in 10 weeks.
Since there is a need, let's implement it. The usual practice is to write an application to grab a remote static resource and feed it back to the front end, which is a simple reverse proxy. However, it is tedious and inefficient to do so. In fact, we can directly proxy these static files through nginx.
nginx's proxy_pass supports filling in any address and dns parsing. So my idea is to convert the original url encryption into the website's own url. Like the one above

http://substack.net/images/browserify_logo.png

Can be encrypted into

764feebffb1d3f877e9e0d0fadcf29b85e8fe84ae4ce52f7dc4ca4b3e05bf1718177870a996fe5804a232fcae5b893ea ( There are many encryption and serialization algorithms on the web, but I won't go into them here )

And put it under our own domain name:


https://ssl.youdomain.com/camo/764feebffb1d3f877e9e0d0fadcf29b85e8fe84ae4ce52f7dc4ca4b3e05bf1718177870a996fe5804a232fcae5b893ea

The steps of decryption will be difficult to achieve with nginx, so when the user passes the above link request, the request will be passed to the decryption program. Here is an example of coffeescript version:

express = require 'express'
app = express()
app.get '/camo/:eurl', (req, res) ->
  {eurl} = req.params
  {camoSecret} = config  # Here you use your own key
  rawUrl = util.decrypt eurl, camoSecret
  return res.status(403).end('INVALID URL') unless rawUrl
  res.set 'X-Origin-Url', rawUrl
  res.set 'X-Accel-Redirect', '/remote'
  res.end()
app.listen 3000

Then write X-Accel-Redirect response header to do the internal jump, and the following steps are completed by nginx.
Here is a complete example of the nginx configuration file:


server {
    listen 80;
    server_name ssl.youdomain.com;
    location /camo/ {
        proxy_pass http://localhost:3000;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_redirect off;
        break;
    }
    location /remote {
        internal;
        resolver 192.168.0.21;  # Have to add dns Server address, otherwise nginx Unable to resolve domain name
        set $origin_url $upstream_http_x_origin_url;
        proxy_pass $origin_url;
        add_header Host "file.local.com";
        break;
    }
}

The upstream module of nginx will save all the response headers with the prefix $upstream_http_ as a variable, so in the above example we put the original url in the X-Origin-Url response header, which becomes $upstream_http_x_origin_url variable in nginx, but it cannot be directly referenced in proxy_pass. Only set can be set to reference, which I don't understand very well, I hope some master can solve it.
That way, every time the user requests it


https://ssl.youdomain.com/camo/764feebffb1d3f877e9e0d0fadcf29b85e8fe84ae4ce52f7dc4ca4b3e05bf1718177870a996fe5804a232fcae5b893ea

When, nginx will grab it

http://substack.net/images/browserify_logo.png

Is returned to the user. We can also add varnish before nginx to cache the contents of static files. So it's even more like githubusercontent.


Related articles: