Build the implementation of Golang application minimum Docker image
- 2020-10-23 20:08:33
- OfStack
I usually use docker to run my golang programs, and here I share 1 of my experiences building docker images. I built the docker image to optimize not only the built volume, but also the build speed.
The sample application
To start with the code example, let's assume that we want to build an http service
package main
import (
"fmt"
"net/http"
"time"
"github.com/gin-gonic/gin"
)
func main() {
fmt.Println("Server Ready")
router := gin.Default()
router.GET("/", func(c *gin.Context) {
c.String(200, "hello world, this time is: "+time.Now().Format(time.RFC1123Z))
})
router.GET("/github", func(c *gin.Context) {
_, err := http.Get("https://api.github.com/")
if err != nil {
c.String(500, err.Error())
return
}
c.String(200, "access github api ok")
})
if err := router.Run(":9900"); err != nil {
panic(err)
}
}
Description:
Gin was chosen as the example to demonstrate that we want to optimize build speed if we have a third party package Line 1 of the main function prints a line of words to illustrate a pit encountered later on startup The time was printed following the route, to demonstrate the pit encountered later about the time zone Routing github try to access https: / / api github. com, behind to demonstrate certificate pit
Here we can try out the volume of the package after the build
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
14.6MB, this is the hello world for 1 http service, of course this is because of the use of gin, so it is a little large, if the standard package net/http hello world written, the volume is approximately 7 MB
The evolution of the Dockerfile
Version 1, preliminary optimization
Let's start with version 1
FROM golang:1.14-alpine as builder
WORKDIR /usr/src/app
ENV GOPROXY=https://goproxy.cn
COPY ./go.mod ./
COPY ./go.sum ./
RUN go mod download
COPY . .
RUN go build -ldflags "-s -w" -o server
FROM scratch as runner
COPY --from=builder /usr/src/app/server /opt/app/
CMD ["/opt/app/server"]
Description:
golang: 1.14-ES55en was chosen as the compilation environment because it is the smallest golang compilation environment GOPROXY was set up to speed up builds First copy go.mod and go.sum, then go mod download to prevent redownloading the dependent package with each build, and use the docker build cache to speed up the build When go build is added with -ES68en "-ES69en-ES70en", the debugging information of the build package is removed, and the volume of the go program after the build is reduced, which can be reduced by about 1/4 Multi-stage build is used, that is, FROM XXX as xxx. When building the package, it USES the image with compilation environment to build. When running, it does not need the compilation environment of go at all, so it USES the empty image of docker to run during the run phase. This is the most efficient way to reduce the volume of the mirror image.
Ok, now let's start building the image
$ docker build -t server .
...
Successfully built 8d3b91210721
Successfully tagged server:latest
At this point, the build is successful and look at the mirror size
$ docker images
server latest 8d3b91210721 1 minutes ago 11MB
11MB, ok, now run 1 under
$ docker run -p 9900:9900 server
standard_init_linux.go:211: exec user process caused "no such file or directory"
The startup error was found, and line 1 of the main function print statement did not appear, so the entire program did not run at all. The error was due to a lack of library dependent files. This is actually a built go program that also relies on the underlying so library file, which you can see after the physical machine has compiled
$ go build -o server
$ ldd server
linux-vdso.so.1 (0x00007ffcfb775000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9a8dc47000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9a8d856000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9a8de66000)
It turns out that there are a couple of dependency libraries, even though they're all at the bottom of the heap, which is what you would expect for a typical operating system, but we chose scratch, and there's really nothing in this image except the linux kernel.
This is because go build is enabled by default CGO, do not believe you can try this command go env CGO_ENABLED, in CGO on, whether the code is used or not CGO, there will be a library dependent file, the solution is also very simple, manually specify CGO off line, and the package volume will not increase oh, will reduce it
$ CGO_ENABLED=0 go build -o server
$ ldd server
not a dynamic executable
Version 2, fix run times error
FROM golang:1.14-alpine as builder
WORKDIR /usr/src/app
ENV GOPROXY=https://goproxy.cn
COPY ./go.mod ./
COPY ./go.sum ./
RUN go mod download
COPY . .
-RUN go build -ldflags "-s -w" -o server
+RUN CGO_ENABLED=0 go build -ldflags "-s -w" -o server
FROM scratch as runner
COPY --from=builder /usr/src/app/server /opt/app/
CMD ["/opt/app/server"]
go build CGO_ENABLED=0
$ docker build -t server .
...
Successfully built a81385160e25
Successfully tagged server:latest
$ docker run -p 9900:9900 server
[GIN-debug] GET / --> main.main.func1 (3 handlers)
[GIN-debug] GET /github --> main.main.func2 (3 handlers)
[GIN-debug] Listening and serving HTTP on :9900
Ok, so let's go to 1 and try it out, check the current time before accessing it
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
0
Find something wrong
The current system time is 13:11:28, but according to the displayed time, it is 05:11:53. In fact, the time zone in docker container is wrong. The default time zone is 0, but our country is zone 8 east Try to access https: / / api. github. com/https site, is it the certificate errorTo solve the problem
Place the root certificate in the container Set the container time zone
Version 3, resolves runtime time zone and certificate issues
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
1
In the builder phase, the ES150en-ES151en tzdata libraries were installed, and in the runner phase, the time zone configuration and root certificate were duplicated
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
2
Go to 1 and try it
$ date
Fri May 29 13:27:16 CST 2020
$ curl http://localhost:9900
hello world, this time is: Fri, 29 May 2020 13:27:16 +0800
$ curl http://localhost:9900/github
access github api ok
1 cut is normal. Look at the current mirror size
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
4
At 11.3MB, it's already pretty small, but it could be even smaller if you compress the built package once more
Version 4, take a step forward to reduce the volume
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
5
In builder phase, installed upx and go build is completed, use upx compression under 1, perform a build, you will find the build time getting longer, it's because I give -- best upx Settings of parameters, which is the biggest level of compression, such compression out will be as small as possible, after if too slow, can reduce the compression level from 1 to 9, the greater the digital compression level is higher, and the slower. I looked at the mirror volume after the --best build was complete.
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
6
This can be small, just 4.26MB, try those two interfaces again, 1 cut normal. That's it for optimization.
The final Dockerfile
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
7
conclusion
To reduce the size of the image, it is important to first build in multiple stages so that you can separate the build environment from the run environment.
In addition, choosing scratch as the mirror is actually unwise. Although it is small, it is too primitive with no tools in it. After the program is launched, you can't even enter the container, and even if you enter it, you can't do anything. Therefore, it is not recommended to choose scratch as the operating environment even if 1 desires to pursue the smallest mirror volume. For the time being, I have only stepped on a small part of the pits, and there are still more pits left behind, so I have no interest in stepping on the pits of scratch.
It is recommended to select alpine. The mirror size of alpine is 5.61. MB is actually the uncompressed image size. In addition, all the mirror volume I mentioned above refers to the uncompressed mirror volume, which is different from the actual upload and download volume. docker will compress the image once and then transmit the image
busybox is a very small image, its size is 1.22MB, download 705.6 KB, most of the linux commands are available, but the environment is still primitive, you can try it
Both alpine and busybox will have the same time zone and certificate issues as described above. Switching to alpine or busybox is as simple as changing the base image of runner
-FROM scratch as runner
+FROM alpine as runner
or
$ go build -o server
$ ls -alh | grep server
-rwxrwxrwx 1 eyas eyas 14.6M May 29 10:26 server
9