Method for creating an Docker container cluster with Docker Swarm and DigitalOcean on Ubuntu 16.04

  • 2020-06-03 08:51:38
  • OfStack

introduce

Docker Swarm is an Docker local solution for deploying Docker host clusters. You can use it to quickly deploy clusters of Docker hosts running on local machines or supported cloud platforms.

Prior to Docker 1.12, setting up and deploying the Docker host cluster required an external key value store (such as etcd or Consul) for service discovery. However, with Docker 1.12, external discovery services are no longer required because Docker provides a built-in key-value store that works out of the box.

In this tutorial, you will learn how to deploy a set of Docker machines using the Swarm capabilities on Docker 1.12. Each Docker node in the cluster will run Ubuntu 16.04. Although you can run a cluster of 10, hundreds, or thousands of Docker hosts, the cluster we will set up in this tutorial will consist of one manager node and two working nodes with three cluster members. After completing this tutorial, you can easily add more nodes to the cluster.

A prerequisite for

For this tutorial, you need:

Local machine with Docker installed. Your local computer can run any Linux distribution, even Windows or macOS. For Windows and macOS, install Docker using the official installer. If you have Ubuntu 16.04 running on your local computer but no Docker installed, see how to install and use Docker on Ubuntu 16.04 for instructions.

DigitalOcean API token. If not, use this guide to generate it. When generating a token, make sure it has read/write scope. This is the default value, so if you don't change any options when you generate it, it will be read-write. To make it easier to use on the command line, be sure to assign tokens to the variables given in this article.

Docker Machine is installed on the local computer and you will use it to create three hosts. On Windows and macOS, the Docker installation includes Docker Machine.

Step 1: Configure the cluster nodes

We need to create several Docker hosts for the cluster. As a refresher, the following command provides a single Docker host, where $DOTOKEN is an environment variable that evaluates to your DigitalOcean API token:


docker-machine create --driver digitalocean --digitalocean-image ubuntu-16-04-x64 --digitalocean-access-token $DOTOKEN machine-name

Imagine that you must set up a cluster of at least three nodes and configure one host at a time.

We can use this command in conjunction with 1 simple Bash script to automate the process of configuring any number of Docker hosts. Execute this command on the local machine to create three Docker hosts named node-1, ES73en-2, and ES74en-3:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done

Upon successful completion of the command, you can verify that all computers have been created by accessing your DigitalOcean dashboard or entering the following command:


docker-machine ls

The output should look like the following and should serve as a quick reference to find the IP address of the node:


Output
NAME ACTIVE DRIVER  STATE URL    SWARM DOCKER ERRORS
node-1 - digitalocean Running tcp://104.236.239.4:2376  v1.12.2 
node-2 - digitalocean Running tcp://104.131.165.210:2376  v1.12.2 
node-3 - digitalocean Running tcp://104.236.76.250:2376  v1.12.2

At this point, all three Docker hosts have been created, and you have the IP address for each host. They are also running Docker 1.12.x, but are not yet part of the Docker cluster. In the next steps, we will configure the firewall rules so that the nodes become cluster members, select one of them as the Docker Swarm manager, and configure the rest as Docker Swarm worker threads.

Step 2: Configure the firewall rules to allow Docker group traffic

The cluster must have at least one node as a manager, but for production Settings, three managers are recommended. For this setting, let's select the first node and set it to the Swarm manager. The other two nodes will be working nodes.

Some network ports must be open on the node that will be part of Cluster 1 for the cluster to work properly. This requires the firewall to be configured to allow traffic through these ports. Because there are three different firewall applications available for this task, the commands you need to execute on the nodes of each firewall application are documented in separate articles. Follow this guide and configure firewalls for each host. Open the appropriate port on the manager, and repeat to open the port on both client nodes.

After you complete this step, you can initialize the cluster manager.

Step 3 initializes the cluster manager

We have decided that node-1 will be our cluster manager, so log in from the local machine to the node:


docker-machine ssh node-1

The command prompt changes to reflect the fact that you are now logged in to that particular node. To configure the node as Swarm manager, type the following command:


docker swarm init --advertise-addr node_ip_address

node_ip_address is the IP address of the node. You can get it from the output of docker ES123en-ES124en ls or from the DigitalOcean dashboard.

You should see output similar to the following:


Output
Swarm initialized: current node (a35hhzdzf4g95w0op85tqlow1) is now a manager.
To add a worker to this swarm, run the following command:
 docker swarm join \
 --token SWMTKN-1-3k7ighcfs9352hmdfzh31t297fd8tdskg6x6oi8kpzzszznffx-6kovxm3akca2qe3uaxtu07fj3 \
 104.236.239.4:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

Is a node in the output ID, in this example as a35hhzdzf4g95w0op85tqlow1, and instructions on how to add other nodes to the cluster information.

So now you have 1 manager configured Docker Swarm. Let's add the remaining nodes as workers.

Step 4 adds the nodes to the cluster

To complete this step, you may want to open another terminal and now leave the terminal TAB or window that you use to individually log in to the Swarm manager.

First, connect to node-2 from the local machine:


docker-machine ssh node-2

This command is then executed, where your_swarm_token is the token received when the cluster was created in the previous step, and manager_node_ip_address is IP for the Swarm manager:


docker swarm join \
--token your_swarm_token \
manager_node_ip_address:2377

When the command executes successfully, you will see this response:


Output
This node joined a swarm as a worker.

Log out of node-2 and repeat this process with node-3 to add it to the cluster.

You have now added two working nodes to the cluster. If the firewall rules are configured correctly, you now have a working Docker Swarm and all the nodes are synchronized.

Step 5. Manage the cluster

After the manager and work nodes are assigned to the cluster, all Docker Swarm administration commands must be executed on the manager node. So go back to the terminal you used to add the manager and type this command to see all the members of the cluster:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
0

The output should resemble:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
1

This output indicates that we are working on a 3-node Docker Swarm and its nodes with one manager and two workers. To see other administrative commands that can be run on the manager node, type:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
2

For more information about clustering, you can use the following command on manager or workers (it is a generic Docker command) :


docker info

The output should be of this type and should indicate the state of the cluster (active or pending), the number of nodes in the cluster, and whether a particular node is a manager or worker.


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
4

If the same command is repeated on the worker thread node, the Is Manager line should appear as false.

Tip: You can add or remove nodes from the cluster at any time. In addition, a work node can be upgraded to a manager, and the manager can be converted to a worker.

Now let's run a service on the cluster.

Step 6: Run the service in Docker Swarm

Now that you have 1 Docker Swarm up and running, let's run 1 test container and see how the manager handles it. On computers running Docker Engine 1.12 or later, the container is deployed as a service using the docker docker service command. Like the docker node command, the docker service command can only be executed on the manager node.

So let's deploy 1 web server service using the official Nginx container images:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
5

In this command, we map port 80 in the Nginx container to port 80 on the cluster so that we can access the default Nginx page from anywhere.

To see the services running on the cluster, type:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
6

The output should take this form. The REPLICAS column shows how many service instances are running:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
7

You can determine the node on which the service is running by using docker service ps followed by the service name.


docker service ps webserver

The output should look like the following:


for i in 1 2 3; do docker-machine create --driver digitalocean \
--digitalocean-image ubuntu-16-04-x64 \
--digitalocean-access-token $DOTOKEN node-$i; done
9

In this example, the webserver service runs on node-1. Since this is an Web server running on the default port, you can access it by pointing your browser to http:// node-1_ip_ES258en. Give it a try. You'll see the default page for Nginx.

With the magic of mesh networks, services running on the nodes can be accessed from any other node in the cluster. For example, this Nginx service can also be accessed by pointing the browser to the IP address of any node in the cluster, not just the IP address of the node it is running on. Give it a try.

Another feature of Docker Swarm is the ability to scale the service, that is, to start other instances of the service. Suppose we want to extend the webserver service we started earlier to five instances. To do this, we simply type the following command and the system will create four instances:


docker service scale webserver=5

And the output of docker docker service ps will show the new instance started on which nodes:


Output
ID    NAME  IMAGE NODE DESIRED STATE CURRENT STATE  ERROR
39yprxsaaekuif951cl0o4wau webserver.1 nginx node-1 Running Running 8 hours ago 
1er2rbrnj6ltanoe47mb653wf webserver.2 nginx node-3 Running Running 14 seconds ago 
evassgyvruh256ebv5pj3bqcz webserver.3 nginx node-3 Running Running 14 seconds ago 
d453agrdpgng47klbl6yfjnka webserver.4 nginx node-1 Running Running 18 seconds ago 
2hsdevx178rg15gqxhzrsnmg6 webserver.5 nginx node-2 Running Running 14 seconds ago

This indicates that two of the four new instances were started on node-3, one on node-1 and one on node-2.

Finally, if the service is down, it will automatically restart on the same node or on a different node if the original node is no longer available.

conclusion

You've seen how easy it is to set up Docker Swarm using Docker Engine 1.12 and the new Swarm mode. You also learned how to perform several administrative tasks on a cluster. But there's more. To see the Docker Swarm commands available, execute the following command on Swarm manager.


docker swarm --help

Related articles: