Use docker buildx Implement multi platform compilation - Case report
In the previous article Use docker buildx Implement multi platform compilation - Environment chapter Describes how to deploy docker buildx Environmental Science .
The author will share several meaningful cases in use
0x00 Say first conclusion
docker buildx
It runs in a container environment , therefore scheduler and builder Native configuration (ex,/etc/hosts
,/etc/docker/daemon.json
) Most of the configurations and scenarios for It's actually not available .- Use
ssh://[email protected]
You can easily perform remote builds , Especially in building the most basic image , When you need an overseas mirror . docker buildx
It's still experimental , Constantly updated . Especially in Dockerfile There are many new features to use , You can refer to dockerfile experimental- It's very convenient to transfer and merge images , combination
github action
Can be fully hosted .
0x01 Distal builder
In the previous article , We go through qemu
Multi architecture simulation of the environment . But actually in use , Execution efficiency is particularly low in a simulation architecture , If you have already performed some project compilation , Then you should feel it .
As the author previously maintained opencv Dual architecture project , adopt github action Cross platform compilation , For three or four hours . Under this framework, only a short period of 10 A few minutes .
therefore , The need to perform compilation or other work under the same architecture is particularly urgent .
- Fortunately
docker buildx
Can support It's special for the special plane - Perform the corresponding tasks according to the machine architecture . - in addition ,
docker
It also supports based onssh://[email protected]
Remote call to protocol . Through password free certificates , Very easy to manage .
execution environment
Suppose there are three machines , And scheduler Can pass the certificate password free login two builder.
purpose | The host address | CPU framework |
---|---|---|
scheduler | 10.100.100.10 | - |
arm64 builder | 192.168.100.101 | arm64 / aarch64 |
amd64 builder | 192.168.100.102 | amd64 / x86_64 |
establish remotebuilder
stay scheduler(10.100.100.10) establish remotebuilder , The order is as follows
ARM64=ssh://[email protected]
AMD64=ssh://[email protected]
## Be careful : Name it here remotebuilder
DOCKER_HOST=${AMD64} docker buildx create --name remotebuilder --node hk-amd64 --platform=amd64
### --append Indicates append , Instead of recreating
DOCKER_HOST=${ARM64} docker buildx create --append --name remotebuilder --node hk-arm64 --platform=arm64
## Use remotebuilder
docker buildx use remotebuilder
## see remotebuilder state
docker buildx ls --builder remotebuilder
This is the place , remote builder
And created successfully .
When we do it locally docker buildx build ...
During the mission , The task will be distributed to the corresponding cpu On the host of the architecture .
Perform a mirror build
- establish Dockerfile
here Dockerfile
It's simple , It's just pulling centos:8
Mirror image .
mkdir -p centos/ && cd centos/
echo "FROM centos:8" > Dockerfile
tree
# .
# └── Dockerfile
# 0 directories, 1 file
- Use
docker buildx
Command pull image .
Be careful : We don't use it here
--tag
Specify the target image name .
docker buildx build --platform=linux/amd64,linux/arm64 .
- Re execution
docker buildx
command .
Be careful : This time we used
--tag
and--push
, After the build is complete , Will be pushed to the targetdocker registry
Be careful 2: If you want to execute
--push
success , Only need scheduler Execute on the hostdocker login
that will do , builder There is no login requirement on .
docker buildx build --platform=linux/amd64,linux/arm64 --tag tangx/centos:8 --push .
Pay attention to the position of the red box , The direct result of this time is , More exporting
and merging
When task execution changes to , Built layer
The cache will exist with remote builder
On , meanwhile scheduler It will save multiple architectures manifest Information .
Output results tangx/centos:8 Can be in dockerhub See above .
0x02 Configuration optimization
Use in China docker buildx
There's one big problem , It's the Internet .
because driver
It's running in the container , Refer to official documentation buildx - github.com, As a result, many of the native configurations do not work .
Such as docker.io
In this way, the official image of foreign countries , The pulling speed is not ideal .
Use mirrors to accelerate optimization
- New configuration file
buildkitd.toml
# buildkitd.toml
[registry."docker.io"]
mirrors = ["wlzfs4t4.mirror.aliyuncs.com"]
And create local builder
### create builder with mirror
docker buildx create --use --name localbuilder --platform=linux/amd64,linux/arm64 --config=buildkitd.toml
- newly build
Dockerfile
# Dockerfile
FROM centos:8
And build a mirror image
### build a image
docker buildx build --platform=linux/amd64,linux/arm64 .
The results are shown in the figure above , It takes about 14s about
Using default parameters , No mirror optimization
- Create... That doesn't use mirror optimization mirror, And build
### without mirror
docker buildx create --use --name localbuilder-no-mirror --platform=linux/amd64,linux/arm64
### build a image
docker buildx build --platform=linux/amd64,linux/arm64 .
You can see from the picture , In the absence of mirror Under the circumstances , There's a network problem .
This is because builder Run in container class , It's not using the host
/etc/docker/daemon.json
To configure ( Suppose the host is already configured mirror), It is equivalent to domestic Direct connectiondocker.io
Pull out the mirror image .
0x03 Dockerfile Case study
If you read Dockerfile ARG
Usage and scope analysis , So the following two Dockerfile
It's easy .
sync
The only effect : Rename the image , The synchronization of non mirror image is realized by parameter .
ARG IMAGE
FROM ${IMAGE}
image=tangx/alpine
tag=3.12
docker buildx build --platform=linux/amd64,linux/arm64 \
--tag ${image}:${tag} \
--file Dockerfile
--build-arg IMAGE=alpine:3.12 \
.
combine
Through here Multistage In the building Alias
And ${TARGETARCH}
The way , Will the two separate tag The images merge into one . for example minio/minio
Mirror image .
ARG TARGETARCH
FROM example.com/alpine:3.12-arm64 as arm64
FROM example.com/alpine:3.12-amd64 as amd64
FROM ${TARGETARCH}
docker buildx build --platform=linux/amd64,linux/arm64 \
--tag example.in/alpine:3.12 \
--file Dockerfile
--build-arg IMAGE=alpine:3.12 \
.
This article starts with the author's blog : https://tangx.in/2020/11/07/docker-buildx-examples/
This article is originally contained in the author's personal blog https://tangx.in
If you find this helpful , You can pay attention to the author's official account of WeChat
Please have a cup of coffee