Solving the Problem of Insufficient Storage Resource Pool in Docker Server

Catalog 1. Problem description
2. Problem analysis
STEP 3 Solve problems
1. View Docker disk usage
2. Execute cleanup commands on recyclable storage resources
Summarize

System environment:

Docker version: 19.03. 13 Operating system version: CentOS 7.8

1. Description of the problem

Recently, when executing the Docker Run Command to start mirroring, the Docker Start Mirroring Command could not be executed normally, prompting the following error message:

Error: Error response from daemon: devmapper: Thin Pool has 163051 free data blocks which is less than minimum required 163840 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior

2. Problem analysis

According to the prompted error message, it is caused by insufficient space in Thin Pool pool of Docker, so it is necessary to create more free space in this pool or change the free space size of the pool by using dm.min_free_space option.

We view the Docker information on the server under 1 with the following command to observe the resource usage:

$ docker info

The contents displayed are as follows:

...
Data Space Total: 507.4GB
Data Space Available: 100.67GB
Metadata Space Used: 100.4MB
Metadata Space Total: 20.147GB
Metadata Space Available: 20.047GB
Thin Pool Minimum Free Space: 100.74GB
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
...

From the above information, it is observed that Thin Pool Minimum Free Space = 100.74 GB and Data Space Available = 100.67, which indicates that the available space of Data Space Available is lower than the minimum required space of Thin Pool, which will cause the image to fail to start normally.

STEP 3 Solve problems

1. View Docker disk usage

We can view Docker disk usage using the following command:

$ docker system df

TYPE   TOTAL  ACTIVE SIZE  RECLAIMABLE
Images   500  12  38.31GB  34.5GB (90%)
Containers  64  21  157MB  4.109MB (2%)
Local Volumes 37  18  0B   0B
Build Cache  0  0  

The following information is listed in total:

TYPE: Resource type; TOTAL: Total number of resources; ACTIVE: Number of active resources; SIZE: Size of resource usage space; RECLAIMABLE: Size of recyclable resources;

Each of these components refers to:

Images: The size of storage space occupied by mirroring; Containers: Size of storage space used by started container Local Volumes: Size of storage space used by locally mounted volumes; Build Cache: The size of storage space occupied by building cache;

If you want to see the system usage in detail, you can use the-v command to see the storage details of each resource usage

2. Execute cleanup commands on recyclable storage resources

Docker provides storage resource reclamation with the following command:

Please see the official document of Docker: docker system prune for details of cleaning command

$ docker system prune

This command cleans up all unused containers, networks, images (suspended and unreferenced), and volumes (optional). After executing this command, wait for 1 period of time, and after reclaiming enough resources, the Docker running image returns to normal.

Summarize