Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

Use docker buildx Implement multi platform compilation - Case report

In the previous article Use docker buildx Implement multi platform compilation - Environment chapter Describes how to deploy docker buildx Environmental Science .

The author will share several meaningful cases in use

0x00 Say first conclusion

1. docker buildx It runs in a container environment , therefore scheduler and builder Native configuration （ex, /etc/hosts, /etc/docker/daemon.json ） Most of the configurations and scenarios for It's actually not available .
2. Use ssh://[email protected] You can easily perform remote builds , Especially in building the most basic image , When you need an overseas mirror .
3. docker buildx It's still experimental , Constantly updated . Especially in Dockerfile There are many new features to use , You can refer to dockerfile experimental
4. It's very convenient to transfer and merge images , combination github action Can be fully hosted .

0x01 Distal builder

In the previous article , We go through qemu Multi architecture simulation of the environment . But actually in use , Execution efficiency is particularly low in a simulation architecture , If you have already performed some project compilation , Then you should feel it .

As the author previously maintained opencv Dual architecture project , adopt github action Cross platform compilation , For three or four hours . Under this framework, only a short period of 10 A few minutes .

therefore , The need to perform compilation or other work under the same architecture is particularly urgent .

1. Fortunately docker buildx Can support It's special for the special plane - Perform the corresponding tasks according to the machine architecture .
2. in addition , docker It also supports based on ssh://[email protected] Remote call to protocol . Through password free certificates , Very easy to manage .

execution environment

Suppose there are three machines , And scheduler Can pass the certificate password free login two builder.

establish remotebuilder

stay scheduler(10.100.100.10) establish remotebuilder , The order is as follows

ARM64=ssh://[email protected]
AMD64=ssh://[email protected]
## Be careful : Name it here remotebuilder
DOCKER_HOST=${AMD64} docker buildx create --name remotebuilder --node hk-amd64 --platform=amd64
### --append Indicates append , Instead of recreating 
DOCKER_HOST=${ARM64} docker buildx create --append --name remotebuilder --node hk-arm64 --platform=arm64
## Use remotebuilder
docker buildx use remotebuilder
## see remotebuilder state 
docker buildx ls --builder remotebuilder

This is the place , remote builder And created successfully .

When we do it locally docker buildx build ... During the mission , The task will be distributed to the corresponding cpu On the host of the architecture .

Perform a mirror build

1. establish Dockerfile

here Dockerfile It's simple , It's just pulling centos:8 Mirror image .

mkdir -p centos/ && cd centos/
echo "FROM centos:8" > Dockerfile
tree
# .
# └── Dockerfile
# 0 directories, 1 file

2. Use docker buildx Command pull image .

Be careful : We don't use it here --tag Specify the target image name .

docker buildx build --platform=linux/amd64,linux/arm64 .

3. Re execution docker buildx command .

Be careful : This time we used --tag and --push, After the build is complete , Will be pushed to the target docker registry

Be careful 2: If you want to execute --push success , Only need scheduler Execute on the host docker login that will do , builder There is no login requirement on .

docker buildx build --platform=linux/amd64,linux/arm64 --tag tangx/centos:8 --push .

Pay attention to the position of the red box , The direct result of this time is , More exporting and merging

When task execution changes to , Built layer The cache will exist with remote builder On , meanwhile scheduler It will save multiple architectures manifest Information .

Output results tangx/centos:8 Can be in dockerhub See above .

0x02 Configuration optimization

Use in China docker buildx There's one big problem , It's the Internet .

because driver It's running in the container , Refer to official documentation buildx - github.com, As a result, many of the native configurations do not work .

Such as docker.io In this way, the official image of foreign countries , The pulling speed is not ideal .

Use mirrors to accelerate optimization

1. New configuration file buildkitd.toml

# buildkitd.toml
[registry."docker.io"]
mirrors = ["wlzfs4t4.mirror.aliyuncs.com"]

And create local builder

### create builder with mirror
docker buildx create --use --name localbuilder --platform=linux/amd64,linux/arm64 --config=buildkitd.toml

2. newly build Dockerfile

# Dockerfile
FROM centos:8

And build a mirror image

### build a image
docker buildx build --platform=linux/amd64,linux/arm64 .

The results are shown in the figure above , It takes about 14s about

Using default parameters , No mirror optimization

1. Create... That doesn't use mirror optimization mirror, And build

### without mirror
docker buildx create --use --name localbuilder-no-mirror --platform=linux/amd64,linux/arm64 
### build a image
docker buildx build --platform=linux/amd64,linux/arm64 .

You can see from the picture , In the absence of mirror Under the circumstances , There's a network problem .

This is because builder Run in container class , It's not using the host /etc/docker/daemon.json To configure ( Suppose the host is already configured mirror), It is equivalent to domestic Direct connection docker.io Pull out the mirror image .

0x03 Dockerfile Case study

If you read Dockerfile ARG Usage and scope analysis , So the following two Dockerfile It's easy .

sync

The only effect : Rename the image , The synchronization of non mirror image is realized by parameter .

ARG IMAGE
FROM ${IMAGE}

image=tangx/alpine
tag=3.12
docker buildx build --platform=linux/amd64,linux/arm64 \
--tag ${image}:${tag} \
--file Dockerfile
--build-arg IMAGE=alpine:3.12 \
.

combine

Through here Multistage In the building Alias And ${TARGETARCH} The way , Will the two separate tag The images merge into one . for example minio/minio Mirror image .

ARG TARGETARCH
FROM example.com/alpine:3.12-arm64 as arm64
FROM example.com/alpine:3.12-amd64 as amd64
FROM ${TARGETARCH}

docker buildx build --platform=linux/amd64,linux/arm64 \
--tag example.in/alpine:3.12 \
--file Dockerfile
--build-arg IMAGE=alpine:3.12 \
.

This article starts with the author's blog : https://tangx.in/2020/11/07/docker-buildx-examples/

This article is originally contained in the author's personal blog https://tangx.in

If you find this helpful , You can pay attention to the author's official account of WeChat

Please have a cup of coffee