x86平台上怎么制作ARM平台镜像？

1,892次阅读
没有评论

共计 12964 个字符，预计需要花费 33 分钟才能阅读完成。

x86平台上怎么制作ARM平台镜像？

最近在整ARM平台上的中间件，以往接触的都是因特尔x86的服务器，在这里记录下镜像制作，制作mysql镜像示例，因为mysql官方没有5.x的arm64/v8镜像

参考官方仓库构建方式改造，此处已ubuntu为基础系统镜像，目前需要解决x86平台如何构建ARM平台镜像？一番搜索后发现可以用buildx这个工具镜像构建，官方链接：https://github.com/docker/buildx，省去了去买arm平台服务器😁

制作镜像流程：

  1. 安装buildx插件
  2. 用ubuntu镜像编译mysql5.7.x源码得到二进制文件
  3. 将编译得到mysql5.7.x二进制文件再用新基础ubuntu镜像制作最终mysql镜像

调整环境

要想使用buildx模拟多平台构建，mac/arm CPU平台似乎默认就有该功能(docker server需要开启experimental)，而x86需要满足以下条件：

  • docker版本>=19.03
  • Linux内核版本>=4.8
  • binfmt-support >= 2.1.7
  • dockers服务端开启experimental特性

升级内核

所以博主这边需要升级下内核

# 升级内核
[root@localhost ~]# rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
[root@localhost ~]# rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
Retrieving http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
Retrieving http://elrepo.org/elrepo-release-7.0-4.el7.elrepo.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:elrepo-release-7.0-4.el7.elrepo  ################################# [100%]
[root@localhost ~]# yum --enablerepo="elrepo-kernel" list --showduplicates | sort -r | grep kernel-ml.x86_64
kernel-ml.x86_64                         6.0.8-1.el7.elrepo            elrepo-kernel
kernel-ml.x86_64                         6.0.7-1.el7.elrepo            elrepo-kernel
[root@localhost ~]# yum --enablerepo="elrepo-kernel" install kernel-ml -y

# 调整grub菜单中内核选项，将刚才的内核调整在首选
[root@localhost ~]# grub2-set-default 0
[root@localhost ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.0.8-1.el7.elrepo.x86_64
Found initrd image: /boot/initramfs-6.0.8-1.el7.elrepo.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-862.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-862.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-ea14c7e6d9ba41a79901fbe4fd310e44
Found initrd image: /boot/initramfs-0-rescue-ea14c7e6d9ba41a79901fbe4fd310e44.img
done

# 升级完内核需要重启
[root@localhost ~]# reboot
[root@localhost ~]# hostnamectl
   Static hostname: localhost.localdomain
         Icon name: computer-vm
           Chassis: vm
        Machine ID: ea14c7e6d9ba41a79901fbe4fd310e44
           Boot ID: 5b3ebaeb4006420bbd902849bddb2b8c
    Virtualization: vmware
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 6.0.8-1.el7.elrepo.x86_64
      Architecture: x86-64

安装docker-ce

[root@localhost ~]# curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun

# 开始实验室特性
[root@localhost ~]# cat >/etc/docker/daemon.json<<-'EOF'
{
        "experimental": true
}
EOF
[root@localhost ~]# systemctl daemon-reload
[root@localhost ~]# systemctl restart docker

# 查看此时版本和特性experimental功能开关
[root@localhost ~]# docker version
Client: Docker Engine - Community
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 18:04:24 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 18:02:38 2022
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.9
  GitCommit:        1c90a442489720eec95342e1789ee8a5e1b9536f
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

下载buildx插件

似乎博主这里目前没有装buildx插件，也有buildx功能，似乎是新版自带了，只是没开启该功能。若读者的未有该功能则下载插件安装

[root@localhost ~]# mkdir -p ~/.docker/cli-plugins/
[root@localhost ~]# wget https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-amd64
[root@localhost ~]# mv buildx-v0.9.1.linux-amd64 ~/.docker/cli-plugins/docker-buildx
[root@localhost ~]# chmod a+x ~/.docker/cli-plugins/docker-buildx

# 此时执行buildx命令测试，它不会自动table补全
[root@localhost ~]# docker buildx

Usage:  docker buildx [OPTIONS] COMMAND

Extended build capabilities with BuildKit

Options:
      --builder string   Override the configured builder instance

Management Commands:
  imagetools  Commands to work on images in registry

Commands:
  bake        Build from a file
  build       Start a build
  create      Create a new builder instance
  du          Disk usage
  inspect     Inspect current builder instance
  ls          List builder instances
  prune       Remove build cache
  rm          Remove a builder instance
  stop        Stop builder instance
  use         Set the current builder instance
  version     Show buildx version information

Run 'docker buildx COMMAND --help' for more information on a command.
[root@localhost ~]# docker buildx version
github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689

配置buildx支持多平台

# 查看目前buildx上所支持的cpu架构平台
[root@localhost ~]# docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS  BUILDKIT PLATFORMS
[root@localhost mysql-5.7.38]# docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS   BUILDKIT PLATFORMS
multiarch *  docker-container
  multiarch0 unix:///var/run/docker.sock inactive
default      docker
  default    default                     running  20.10.21 linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

# 配置支持多cpu架构平台模拟器
[root@localhost ~]# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon

# 创建一个构建器
[root@localhost ~]# docker buildx create --name multiarch --driver docker-container --use
multiarch

查看当前构建器
[root@localhost ~]# docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS   BUILDKIT PLATFORMS
multiarch *  docker-container
  multiarch0 unix:///var/run/docker.sock inactive
default      docker
  default    default                     running  20.10.21 linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

普通方式构建镜像

ARM服务器上构建

参考官方dockerfile修改后，为了方便构建测试，先完成源码编译功能部分

# 相关文件已放置git仓库
[root@localhost ~]# git clone https://gitea.xadocker.cn/container-image/mysql.git
[root@localhost ~]# cd mysql
[root@localhost mysql]# ll mysql-5.7.38/
total 60
-rwxr-xr-x 1 root root 16145 Nov 14 20:18 docker-entrypoint.sh
-rw-r--r-- 1 root root  2049 Nov 14 20:41 dockerfile.2image
-rw-r--r-- 1 root root   951 Nov 14 20:40 dockerfile.bin2image.ubuntu
-rw-r--r-- 1 root root 36039 Nov 14 20:18 install-db.sh

[root@localhost mysql]# cd mysql-5.7.38/
[root@localhost mysql-5.7.38]# cat dockerfile.bin2image.ubuntu
FROM ubuntu:20.10
MAINTAINER 1793360097@qq.com
# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r mysql && useradd -r -g mysql mysql
COPY install-db.sh /usr/local/bin/
RUN bash /usr/local/bin/install-db.sh && rm -rf /usr/local/src/* && rm -rf /usr/local/mysql/bin/{mysqltest_embedded,mysql_embedded,mysql_client_test_embedded} /usr/local/mysql/mysql-test
RUN mkdir -p /etc/mysql/conf.d/ && echo '[mysqld]\nskip-host-cache\nskip-name-resolve' > /etc/mysql/conf.d/docker.cnf \
        && rm -rf /var/lib/mysql && mkdir -p /var/lib/mysql/share /var/run/mysqld \
        && chown -R mysql:mysql /var/lib/mysql /var/run/mysqld \
# ensure that /var/run/mysqld (used for socket and lock files) is writable regardless of the UID our mysqld instance ends up having at runtime
        && chmod 1777 /var/run/mysqld /var/lib/mysql

# 开始构建二进制包
[root@localhost mysql-5.7.38]# docker build -f dockerfile.bin2image.ubuntu . -t  mysql-bin:5.7.38-ubuntu

# 构建出来的镜像好大。。。对比了下官方的才0.5G，不知道少了啥
[root@localhost mysql-5.7.38]# docker images
REPOSITORY      TAG                          IMAGE ID       CREATED          SIZE
mysql           mysql-bin:5.7.38-ubuntu      05ba82071e80   22 minutes ago   2.93GB
ubuntu          20.10                        2f8908083042   15 months ago    71.8MB

使用多阶段构建的方式打镜像

[root@localhost mysql-5.7.38]# cat dockerfile.2image
FROM ubuntu:20.10 as builder
MAINTAINER 1793360097@qq.com
# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r mysql && useradd -r -g mysql mysql
COPY install-db.sh /usr/local/bin/
RUN bash /usr/local/bin/install-db.sh && rm -rf /usr/local/src/* && rm -rf /usr/local/mysql/bin/{mysqltest_embedded,mysql_embedded,mysql_client_test_embedded} /usr/local/mysql/mysql-test
RUN mkdir -p /etc/mysql/conf.d/ && echo '[mysqld]\nskip-host-cache\nskip-name-resolve' > /etc/mysql/conf.d/docker.cnf \
        && rm -rf /var/lib/mysql && mkdir -p /var/lib/mysql/share /var/run/mysqld \
        && chown -R mysql:mysql /var/lib/mysql /var/run/mysqld \
# ensure that /var/run/mysqld (used for socket and lock files) is writable regardless of the UID our mysqld instance ends up having at runtime
        && chmod 1777 /var/run/mysqld /var/lib/mysql

FROM ubuntu:20.10
COPY --from=builder /usr/local/mysql/* /usr/local/mysql
RUN groupadd -r mysql && useradd -r -g mysql mysql && mkdir -p /var/lib/mysql/share /var/run/mysqld /docker-entrypoint-initdb.d && chown mysql:mysql -R /usr/local/mysql /var/lib/mysql /var/run/mysqld /docker-entrypoint-initdb.d

ENV GOSU_VERSION 1.14
RUN set -eux; \
# TODO find a better userspace architecture detection method than querying the kernel
        arch="$(uname -m)"; \
        case "$arch" in \
                aarch64) gosuArch='arm64' ;; \
                x86_64) gosuArch='amd64' ;; \
                *) echo >&2 "error: unsupported architecture: '$arch'"; exit 1 ;; \
        esac; \
        curl -fL -o /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$gosuArch.asc"; \
        curl -fL -o /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$gosuArch"; \
        export GNUPGHOME="$(mktemp -d)"; \
        gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
        gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
        rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
        chmod +x /usr/local/bin/gosu; \
        gosu --version; \
        gosu nobody true

VOLUME /var/lib/mysql
ENV MYSQL_MAJOR 5.7
ENV MYSQL_VERSION 5.7.38-ubuntu20.10
COPY docker-entrypoint.sh /usr/local/bin/
RUN ln -s usr/local/bin/docker-entrypoint.sh /entrypoint.sh # backwards compat
ENTRYPOINT ["docker-entrypoint.sh"]

EXPOSE 3306 33060
CMD ["mysqld"]

[root@localhost mysql-5.7.38]# docker build -f dockerfile.2image . -t  mysql:5.7.38-ubuntu

使用buildx构建ARM平台镜像

# 使用模拟器构建arm平台镜像
[root@localhost mysql-5.7.38]# \cp dockerfile.2image dockerfile
[root@localhost mysql-5.7.38]# docker buildx build --platform linux/arm64 -t 5.7.38-ubuntu.10 . --load
[+] Building 245.5s (8/11)
 => [internal] booting buildkit                                                                                                                                                                                                                    8.0s
 => => pulling image moby/buildkit:buildx-stable-1                                                                                                                                                                                                 7.2s
 => => creating container buildx_buildkit_multiarch0                                                                                                                                                                                               0.8s
 => [internal] load build definition from dockerfile                                                                                                                                                                                               0.0s
 => => transferring dockerfile: 1.09kB                                                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.10                                                                                                                                                                                    5.0s
 => [1/6] FROM docker.io/library/ubuntu:20.10@sha256:a7b08558af07bcccca994b01e1c84f1d14a2156e0099fcf7fcf73f52d082791e                                                                                                                             22.1s
 => => resolve docker.io/library/ubuntu:20.10@sha256:a7b08558af07bcccca994b01e1c84f1d14a2156e0099fcf7fcf73f52d082791e                                                                                                                              0.0s
 => => sha256:06dd9de3e7a96bf4bd49de5d681cb7062c5b3012c0f1b1bcd9d059380f55a1d2 29.88MB / 29.88MB                                                                                                                                                  20.7s
 => => extracting sha256:06dd9de3e7a96bf4bd49de5d681cb7062c5b3012c0f1b1bcd9d059380f55a1d2                                                                                                                                                          1.4s
 => [internal] load build context                                                                                                                                                                                                                  0.0s
 => => transferring context: 36.50kB                                                                                                                                                                                                               0.0s
 => [2/6] RUN groupadd -r mysql && useradd -r -g mysql mysql && echo > /etc/apt/sources.list                                                                                                                                                       0.8s
 => [3/6] COPY install-db.sh /usr/local/bin/                                                                                                                                                                                                       0.0s
 => [4/6] RUN bash /usr/local/bin/install-db.sh Modify_Source DB_Dependent                                                                                                                                                                      245.5s
 => => # Get:37 http://old-releases.ubuntu.com/ubuntu groovy-security/main arm64 libasan6 arm64 10.3.0-1ubuntu1~20.10 [2024 kB]
 => => # Get:38 http://old-releases.ubuntu.com/ubuntu groovy-security/main arm64 liblsan0 arm64 10.3.0-1ubuntu1~20.10 [800 kB]
 => => # Get:39 http://old-releases.ubuntu.com/ubuntu groovy-security/main arm64 libtsan0 arm64 10.3.0-1ubuntu1~20.10 [1972 kB]
 => => # Get:40 http://old-releases.ubuntu.com/ubuntu groovy-security/main arm64 libubsan1 arm64 10.3.0-1ubuntu1~20.10 [764 kB]
 => => # Get:41 http://old-releases.ubuntu.com/ubuntu groovy-security/main arm64 libgcc-10-dev arm64 10.3.0-1ubuntu1~20.10 [906 kB]
 => => # Get:42 http://old-releases.ubuntu.com/ubuntu groovy-security/main arm64 gcc-10 arm64 10.3.0-1ubuntu1~20.10 [15.8 MB]

.......

# 构建出来的镜像太大了，2G+......
root@iZwz9cu8gh0lswws7x1lvqZ:~/mysql/mysql-5.7.38# docker images
REPOSITORY                                       TAG                IMAGE ID       CREATED         SIZE
mysql                                            5.7.38-ubuntu.10   70bdcc26ad31   2 hours ago     2.76GB
gitea.xadocker.cn/container-image/mysql-arm64    5.7.38             70bdcc26ad31   2 hours ago     2.76GB

编译太慢慢慢慢慢慢慢慢了，也可能博主的电脑拉跨，编译了12238.5s，本身就是VM=>docker=>qemu模拟器=》arm64，虚拟化嵌套，性能折上折，还是直接找台arm服务器编译，速度嘎嘎就上来了。嫌慢的读者可以选个简单的中间件去构建，比如nginx/redis等~

留个坑：用jenkins+jnlp agent，通过label选取不同平台进行自动构建。。。又或者可以用gitea+drone的方式，后面有时间实现了再写。。。。

正文完
docker mysql
发表至： 数据库 运维
2022-11-14
 2
xadocker
版权声明：本站原创文章，由 xadocker 2022-11-14发表，共计12964字。
转载说明：除特殊说明外本站文章皆由CC-4.0协议发布，转载请注明出处。
Mongodb 副本集
x86平台上怎么制作ARM平台镜像？
redis内存分析工具redis-rdb-tools
Mongodb 初探
评论（没有评论）