Docker Multiple containers cannot have the same port number

Problem background

In docker, four containers are created with the same mirror image, the network chooses bridge mode, and A service uses the same port number (6000) in all four containers; In order to reduce the number of ports exposed to the outside world, nginx is used as proxy for these four service instances, and the four service instances belong to four upstream, and paths similar to the/service1 and/service2 are used to access the four instances.

At this time, if you access any 1 service locally, you will report a 502 error, which is puzzling.

connect() failed (111: Connection refused) while connecting to upstream

compose file

version: '2'
networks:
 nn:
  driver: bridge
services:
 service-1:
  container_name: service-1
  image: foo
  networks:
   - nn
  volumes:
   - ./logs/1:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/1.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000
  
 service-2:
  container_name: service-2
  image: foo
  networks:
   - nn
  volumes:
   - ./logs/2:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/2.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000
 
 service-3:
  container_name: service-3
  image: foo
  networks:
   - nn
  volumes:
   - ./logs/3:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/3.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000
 
 service-4:
  container_name: service-4
  image: foo
  networks:
   - nn
  volumes:
   - ./logs/4:/apps/aaa/bbb-logs
   - ./common:/apps/aaa/bbb
   - ./xxx/4.xml:/ccc/targets.xml
  entrypoint: foo.sh
  command: start app=foo port=6000  
 
 nginx:
  container_name: nginx
  image: nginx:1.15-alpine
  ports:
   - 6001:6001
  networks:
   - nn
  volumes:
   - ./nginx/nginx.conf:/etc/nginx/nginx.conf
   - ./logs/nginx:/var/log/nginx

nginx.conf

worker_processes 8;
worker_rlimit_nofile 65535; 
events {
    use epoll;
    worker_connections 65535;
 }
 
http {
    include mime.types;
    default_type aplication/octet-stream;
    sendfile on;
    log_format main '[$time_local]$remote_addr-$upstream_addr "$request" $status $body_bytes_sent';
 
    upstream service1.local {
      server service-1:6000;
    }
    upstream service2.local {
     server service-2:6000;
    }
    upstream service3.local {
      server service-3:6000;
    }
    upstream service4.local {
      server service-4:6000;
    }
 
    server {
      listen 6001;
      client_max_body_size 100M;
      proxy_set_header Host $host:$server_port;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
      location /service1/ {
        proxy_pass http://service1.local/;
      }
      location /service2/ {
        proxy_pass http://service2.local/;
      }
      location /service3/ {
        proxy_pass http://service3.local/;
      }
      location /service4/ {
        proxy_pass http://service4.local/;
      }
      location /nginx_status {
        stub_status on;
        access_log off;
      }
    }
}
 

At this time, curl localhost: 6001/service1/api/v1/... will report the above 502 error. It stands to reason that every container has its own network card, and the port numbers of different containers should not conflict.

Solutions

There is no better solution for the time being, so we can only use different port numbers for 4 services, and nginx is modified accordingly.

Supplement: Deploying multiple sets of docker containers on the same server, port redirection problem

In the generation environment, deploy multiple containers and access multiple ports;

For example:-p 80: 80-p 81: 81

When 81 address exits, access the address of port 80 directly.

Misunderstanding: At first, I thought it was a problem with cookie, because cookie was refreshed (cookie does not distinguish port numbers)

Eventually found the cause: redirect problem, because exit redirects to login page

Solution: Configure nginx parameters

proxy_set_header HOST $host; Change to proxy_set_header HOST $host: 81;

Because in any case, the request parameter throws come with the port number.

There is another online method: modify the proxy_redirect parameter (but it didn't work)