A clever combination of Docker and Golang

A clever combination of Docker and Golang

Editor's note: this is a short list of tips and tricks for making Docker more useful when using the Go language. For example, how to compile Go code using different versions of the Go toolchain, and how to cross-compile to different platforms (and test the results!) Or how to make really small container images.

The following article assumes that you already have Docker installed. It doesn't have to be the latest version (this article won't use any fancy Docker features).
Go without go

. "Go can be used without installing go"

If you write Go code, or if you have a slight interest in Go, you're sure to have the Go compiler and Go toolchain installed, so you might want to know, "what's the point?" ; But in some cases, you don't want to install Go to compile Go.

The machine still has the old version Go 1.2 (you can't or don't want to update it), has to use this code base, and requires a higher version of the toolchain. You want to use the cross-compilation feature of Go1.5 (for example, make sure you can create a base 2 file for the operating system X from an Linux system). You want to have multiple versions of Go, but you don't want to completely mess up the system. Want to 100% identify the project and all its dependencies, download, build and run on a pure system.

If you encounter the above situation, find Docker to solve it!

Compile a program in a container

When you install the Go, you can perform go get - v github. com/user/repo to download, to create and install a library. (-v is just information display, if you like the tool chain to run fast and silently, you can remove it!)

You can also perform go get github. com/user/repo /... To download, create, and install everything in repo (including libraries and base 2 files).

We can do this in one container!

Try this:

docker run golang go get -v github.com/golang/example/hello/...

This will pull the golang image (unless you already have one, which will start immediately) and create a container based on it. In that container, go will download an example of "hello world," create it, and install it. But it will install it in this container... How do we run that program now?

Run the program in the container

One way is to submit the container we just created, that is, package it into a new image:

docker commit $(docker ps -lq) awesomeness

Note: docker ps, lq outputs ID (only ID!) of the last container executed. . If you are a unique user of the machine and you have not created another container since the last command, this container is an example of the "hello world" you just created.

Now you can use the image you just built to create the container to run the program:

docker run awesomeness hello

The output will be Hello, Go examples! .

sparkle

When building an image with docker commit, you can specify any Dockerfile command with the --change id. For example, you can use an CMD or ENTRYPOINT command so that docker run awesomeness automatically executes hello.

Run on a primary container

What if you don't want to create an extra image and just want to run this Go program?

Use:

docker run --rm golang sh -c \
"go get github.com/golang/example/hello/... && exec hello"

Wait, what are those fancy things?

--rm tells Docker CLI1 to automatically issue an docker rm command once the container exits. That way, nothing is left. Use the shell logical operator & & Join the create step (go get) and the execute step (exec hello) in 1 step. If you don't like shell, & & It means "and". It allows part 1 go get... And if (and only if!) That part ran successfully, and it will execute part 2 (exec hello). If you're wondering why: it's like a lazy and calculator, only counting the right-hand side if the left-hand value is true. Pass the command to sh, c, because if it is easy to do docker run golang " go get... & & hello & quot; , Docker will try to execute a program named go SPACE get SPACE etc. And that won't work. So, we start an shell and have shell execute the command sequence. Use exec hello instead of hello: this will use the hello program instead of the current process (shell we just started). This ensures that hello is PID 1 in the container. Instead of shell, PID 1 and hello as a child process. This is useless for this trivial example, but when running more useful programs, it will allow them to receive external signals correctly, since the external signals are sent to PID 1 in the container. You might think, what signal? A good example is docker stop, which sends SIGTERM to PID 1 of the container.

Use a different version of Go

When using golang mirroring, Docker is extended to golang:latest, mapping (as you might guess) to the latest version available on Docker Hub.

If you want to use a specific version of Go, it's easy: label it with that version after the mirror name.

For example, if you want to use Go 1.5, modify the above example and replace golang with golang:1.5:

docker run --rm golang:1.5 sh -c \
"go get github.com/golang/example/hello/... && exec hello"

You can see all available versions (and variables) on Docker Hub's Golang mirror page.

Install on the system

Okay, what if you want to run a compiled program on your system instead of a container? We will copy the compiled binary file out of the container. Note that this only works if the container schema and the host schema match; In other words, if you run Docker on Linux. (I'm ruling out someone running the Windows container!)

The easiest way to get a binary file outside the container is to map the $GOPATH/bin directory to a local directory. In the golang container, $GOPATH is /go.

docker run -v /tmp/bin:/go/bin \
golang go get github.com/golang/example/hello/...
/tmp/bin/hello

If you are on Linux, you will see Hello, Go examples! The message. But if it is, for example, on Mac, you might see:
-bash:

/tmp/test/hello: cannot execute binary file

What can we do?

cross-compilation

Go 1.5 has excellent out-of-the-box cross-compilation capabilities, so if your container operating system and/or architecture don't match your system, it's not a problem at all!

To enable cross-compilation, you need to set GOOS and/or GOARCH.

For example, assume that on 64-bit Mac:

docker run -e GOOS=darwin -e GOARCH=amd64 -v /tmp/crosstest:/go/bin \
golang go get github.com/golang/example/hello/...

The cross-compiled output is not directly in $GOPATH/bin, but in $GOPATH/bin/$GOOS_$GOARCH. In other words, to run the program, execute /tmp/crosstest/darwin_amd64/hello.

Install directly to $PATH

If you are on Linux, you can even install it directly to the system bin directory:

docker run -v /usr/local/bin:/go/bin \
golang get github.com/golang/example/hello/...

However, on Mac, trying to use /usr as one volume will not mount Mac's filesystem to the container. The Moby VM (small Linux VM is hidden behind the Docker icon in the toolbar) /usr directory will be mounted. The Docker for Mac version can be customized to set the mount path.

But you can use /tmp or some other directory in your home directory and copy from here.

Creating a dependent image

The Go2 files we produce using this technique are statically linked. This means that all the code that needs to be run, including all the dependencies, is embedded. Dynamically linked programs, in contrast, do not contain some basic libraries (like "libc") and use system-wide replication, which is determined at run time.

This means you can drop the Go compiled program in the container, nothing more, and it will run.

Let's try!

scratch mirror

The Docker ecosystem has a special mirror: scratch. This is an empty mirror. It does not need to be created or downloaded because it is defined as empty.

Create a new empty directory for the new Go dependent image.

In this new directory, create the following Dockerfile:

FROM scratch
COPY ./hello /hello
ENTRYPOINT ["/hello"]

This means: starting with scratch (1 empty image), adding the hello file to the root directory of the image, * defining the hello program as the default program to run after starting the container.

The hello2 base file is then generated as follows:

docker run -v $(pwd):/go/bin --rm \
golang go get github.com/golang/example/hello/...

Note: there is no need to set GOOS and GOARCH, because you want a binary file running in the Docker container, not on the host. So you don't have to set these variables!

Then, create the image:

docker build -t hello .

Test it:

docker run hello

(" Hello, Go examples! ") )

Last but not least, check the size of the mirror:
docker images hello

If the 1 cut is done correctly, the mirror image is approximately 2M. Quite good!
Build things without pushing them to Github

Of course, if you had to push to GitHub, you'd waste a lot of time every time you compiled.

To work on a piece of code and create it in a container, mount a local directory to /go in the golang container. So $GOPATH is a persistent call: docker run-v $HOME/go:/go golang...

But you can also mount the local directory to a specific path to "reload" some packages (those edited locally). Here's a complete example:

# Adapt the two following environment variables if you are not running on a Mac
export GOOS=darwin GOARCH=amd64
mkdir go-and-docker-is-love
cd go-and-docker-is-love
git clone git://github.com/golang/example
cat example/hello/hello.go
sed -i .bak s/olleH/eyB/ example/hello/hello.go
docker run --rm \
-v $(pwd)/example:/go/src/github.com/golang/example \
-v $(pwd):/go/bin/${GOOS}_${GOARCH} \
-e GOOS -e GOARCH \
golang go get github.com/golang/example/hello/...
./hello
# Should display "Bye, Go examples!" 

Network packets and CGo are special cases

Before entering the real Go code world, it must be admitted that there is a slight deviation on the base 2 file. If you are using CGo, or if you are using the net package, the Go linker will generate a dynamic library. In this case, the net package (which does have a lot of useful Go programs in it!) , the culprit is DNS parsing. Most systems have a fancy, modular name resolution system (like name service switching) that relies on plug-ins and, technically, dynamic libraries. By default, Go will try to use it; In this way, it will produce a dynamic library.

How do we solve this?

Reuse another version of libc

One solution is to use a base image that has the necessary libraries for the functionality of the program. Almost any "normal" Linux distribution based on GNU libc will do. So, for example, use FROM debian or FROM fedora instead of FROM scratch. Now the mirror image is going to be a little bit bigger than it was; But at the very least, that extra point will be Shared with the rest of the system.

Note: Alpine cannot be used in this case, because Alpine USES the musl library instead of GNU libc.

Use your own libc

Another solution is to surgically extract the required files and replace them with COPY. You end up with a smaller container. However, this extraction process is difficult and tedious, with too many deeper details to deal with.

If you want to see for yourself, take a look at the ldd and name service switch plug-in mentioned earlier.

Generate static base 2 files with netgo

We can also instruct Go not to use libc of the system, but to use local DNS parsing instead of netgo of Go.

To use it, simply add -tags netgo-installsuffix netgo in the go get option.

-tags netgo indicates that the tool chain USES netgo.

-installsuffix netgo ensures that the result library (any) is replaced by a different, non-default directory. This is avoided if you make multiple calls to go get (or go build)

Conflict between code creation and netgo. If it were created in a container, as we've seen so far, it wouldn't be necessary. There will never be any other Go code to compile in this container. But it's a good idea to get used to it, or at least know that the sign exists.

Special case of SSL certificate

One more thing you'll worry about is that your code must validate the SSL certificate; For example, connect the external API via HTTPS. In this case, you need to put the root certificates into the container as well, because Go does not bundle them into a binary file.
Install the SSL certificate

Again, there are many options available, but the easiest is to use a package that already exists in the release.

Alpine is a good choice because it is very small. Dockerfile below will give you a small base image, but bundled with an expired certificate:

FROM alpine:3.4
RUN apk add --no-cache ca-certificates apache2-utils

Check it out. The mirror image is only 6MB!

Note: -- the no-cache option tells apk (Alpine package manager) to get a list of available packages from Alpine's mirror publication, not on disk. You might see Dockerfiles doing something like this & & apt-get install ... & & rm - rf/var/cache/apt / *; This implements (that is, does not retain the package cache in the final image) something equivalent to a single 1 flag.

An added bonus: putting your application in an Alpine mirroring container gives you a bunch of useful tools. If you need to, you can now put shell into the container and do something with it while it's running.

packaging

We saw how Docker helped us compile Go code in a clean, self-contained environment; How to use different versions of the Go toolchain; And how to cross-compile between different operating systems and platforms.

We also saw how Go helped us create small, container-dependent images for Docker, and described some of the subtle connections between static libraries and network dependencies (no other meaning).

In addition to the fact that Go is really suitable for the Docker project, we would like to show you how Go and Docker learn from each other and work well together!
Thank you

This was first suggested on hacker day 2016, GopherCon.

I would like to thank all those who proofread the material and made Suggestions and comments to make it better, including but not limited to:

Aaron Lehmann
Stephen Day
AJ Bowen

All the mistakes and spelling mistakes are my own; All the good stuff is theirs!

Thank you for reading, I hope to help you, thank you for your support of this site!