Tekton CI 之实战篇（二): DinD方式构建镜像

其他 2020-08-05 10:34:07 阅读次数: 0

Tekton CI 之实战篇（二）： DinD方式构建镜像

背景

简单介绍下什么是dind？使用场景是什么？
DinD即 Docker inside Docker, DinD在容器里有一个完整的docker构建系统,可直接在容器中完成镜像的构建,与之相对应的就是DooD ,通过挂载宿主机的docker.sock文件，调用宿主机的docker daemon去构建镜像。他们的主要使用场景有很多，比较常见的就是CICD场景中了，CICD需要构建镜像。我之前的文章使用的就是DooD的方式，通过挂载宿主机的docker.sock文件。而本讲介绍DinD，进行镜像构建并推送。简单说下DooD和Dind的优势与劣势：

DooD DinD
优势 同一台宿主机上的缓存可以通过同一个 Docker daemon 共享，这样构建镜像会比较快 1、明显支持并发，一个容器实例使用一个docker daemon进程； 2、容器实例之间相互隔离，不干扰，不使用缓存，比较干净 DinD劣势：
劣势 1、由于不同容器实例挂在同一个宿主机的 Docker daemon 进程，所有实例里 docker 命令的权限也是共享的，也就是说不同容器实例可以查看甚至更新、删除到同一个 Docker daemon 下别的容器实例构建产生的镜像。这就是个安全问题； 2、DooD的并行效果不咋地，因为多个容器实例使用同一个Docker daemon进程。可能存在排队或者争抢。 3、DooD的方式会导致宿主机本地的存储被大量消耗，需要定期清理镜像；4、环境隔离性问题，不同容器构建镜像容易出现名字冲突问题 速度上还是没有DooD快，因为不能使用缓存。每个容器构建完后即销毁。

本讲想使用DinD的方式，因为我在实际开发中需要一个相互之间隔离，且每次使用镜像时，都需要重新拉取的环境，所以我选在DinD的方式来构建镜像。

DinD实战

制作DinD镜像

先去dockerhub上找下docker镜像，里面有dind的版本。然后你可以参考tekton的dind-sidecar来构建镜像（参照参考文章[3]），但是由于构建镜像时需要拉取dockerhub镜像作为基础镜像，就需要加上镜像加速器，不然可能会拉取镜像失败导致构建镜像不成功。所以,我们需要构建一个带镜像加速器的docker:dind的镜像。参照 docker镜像的github地址.你可以找到dind的构建所需要的Dockerfile和entrypoint.sh文件。下面我贴一下我的Dockerfile和entrypoint.sh文件：
Dockerfile:(就改了基础镜像，改成了18.05，其他都没改)

FROM docker:18.05
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN set -eux; \
        apk add --no-cache \
                btrfs-progs \
                e2fsprogs \
                e2fsprogs-extra \
                iptables \
                openssl \
                shadow-uidmap \
                xfsprogs \
                xz \
# pigz: https://github.com/moby/moby/pull/35697 (faster gzip implementation)
                pigz \
        ; \
# only install zfs if it's available for the current architecture
# https://git.alpinelinux.org/cgit/aports/tree/main/zfs/APKBUILD?h=3.6-stable#n9 ("all !armhf !ppc64le" as of 2017-11-01)
# "apk info XYZ" exits with a zero exit code but no output when the package exists but not for this arch
        if zfs="$(apk info --no-cache --quiet zfs)" && [ -n "$zfs" ]; then \
                apk add --no-cache zfs; \
        fi

# TODO aufs-tools

# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -x \
        && addgroup -S dockremap \
        && adduser -S -G dockremap dockremap \
        && echo 'dockremap:165536:65536' >> /etc/subuid \
        && echo 'dockremap:165536:65536' >> /etc/subgid

# https://github.com/docker/docker/tree/master/hack/dind
ENV DIND_COMMIT 37498f009d8bf25fbb6199e8ccd34bed84f2874b

RUN set -eux; \
        wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
        chmod +x /usr/local/bin/dind

COPY dockerd-entrypoint.sh /usr/local/bin/

VOLUME /var/lib/docker
EXPOSE 2375 2376

ENTRYPOINT ["dockerd-entrypoint.sh"]
CMD []

entrypony.sh:(增加了阿里云镜像加速地址)

#!/bin/sh
set -eu

_tls_ensure_private() {
        local f="$1"; shift
        [ -s "$f" ] || openssl genrsa -out "$f" 4096
}
_tls_san() {
        {
                ip -oneline address | awk '{ gsub(/\/.+$/, "", $4); print "IP:" $4 }'
                {
                        cat /etc/hostname
                        echo 'docker'
                        echo 'localhost'
                        hostname -f
                        hostname -s
                } | sed 's/^/DNS:/'
                [ -z "${DOCKER_TLS_SAN:-}" ] || echo "$DOCKER_TLS_SAN"
        } | sort -u | xargs printf '%s,' | sed "s/,\$//"
}
_tls_generate_certs() {
        local dir="$1"; shift

        # if ca/key.pem || !ca/cert.pem, generate CA public if necessary
        # if ca/key.pem, generate server public
        # if ca/key.pem, generate client public
        # (regenerating public certs every startup to account for SAN/IP changes and/or expiration)

        # https://github.com/FiloSottile/mkcert/issues/174
        local certValidDays='825'

        if [ -s "$dir/ca/key.pem" ] || [ ! -s "$dir/ca/cert.pem" ]; then
                # if we either have a CA private key or do *not* have a CA public key, then we should create/manage the CA
                mkdir -p "$dir/ca"
                _tls_ensure_private "$dir/ca/key.pem"
                openssl req -new -key "$dir/ca/key.pem" \
                        -out "$dir/ca/cert.pem" \
                        -subj '/CN=docker:dind CA' -x509 -days "$certValidDays"
        fi

        if [ -s "$dir/ca/key.pem" ]; then
                # if we have a CA private key, we should create/manage a server key
                mkdir -p "$dir/server"
                _tls_ensure_private "$dir/server/key.pem"
                openssl req -new -key "$dir/server/key.pem" \
                        -out "$dir/server/csr.pem" \
                        -subj '/CN=docker:dind server'
                cat > "$dir/server/openssl.cnf" <<-EOF
                        [ x509_exts ]
                        subjectAltName = $(_tls_san)
                EOF
                openssl x509 -req \
                                -in "$dir/server/csr.pem" \
                                -CA "$dir/ca/cert.pem" \
                                -CAkey "$dir/ca/key.pem" \
                                -CAcreateserial \
                                -out "$dir/server/cert.pem" \
                                -days "$certValidDays" \
                                -extfile "$dir/server/openssl.cnf" \
                                -extensions x509_exts
                cp "$dir/ca/cert.pem" "$dir/server/ca.pem"
                openssl verify -CAfile "$dir/server/ca.pem" "$dir/server/cert.pem"
        fi

        if [ -s "$dir/ca/key.pem" ]; then
                # if we have a CA private key, we should create/manage a client key
                mkdir -p "$dir/client"
                _tls_ensure_private "$dir/client/key.pem"
                chmod 0644 "$dir/client/key.pem" # openssl defaults to 0600 for the private key, but this one needs to be sared with arbitrary client contexts
                openssl req -new \
                                -key "$dir/client/key.pem" \
                                -out "$dir/client/csr.pem" \
                                -subj '/CN=docker:dind client'
                cat > "$dir/client/openssl.cnf" <<-'EOF'
                        [ x509_exts ]
                        extendedKeyUsage = clientAuth
                EOF
                openssl x509 -req \
                                -in "$dir/client/csr.pem" \
                                -CA "$dir/ca/cert.pem" \
                                -CAkey "$dir/ca/key.pem" \
                                -CAcreateserial \
                                -out "$dir/client/cert.pem" \
                                -days "$certValidDays" \
                                -extfile "$dir/client/openssl.cnf" \
                                -extensions x509_exts
                cp "$dir/ca/cert.pem" "$dir/client/ca.pem"
                openssl verify -CAfile "$dir/client/ca.pem" "$dir/client/cert.pem"
        fi
}

# no arguments passed
# or first arg is `-f` or `--some-option`
if [ "$#" -eq 0 ] || [ "${1#-}" != "$1" ]; then
        # set "dockerSocket" to the default "--host" *unix socket* value (for both standard or rootless)
        uid="$(id -u)"
        if [ "$uid" = '0' ]; then
                dockerSocket='unix:///var/run/docker.sock'
        else
                # if we're not root, we must be trying to run rootless
                : "${XDG_RUNTIME_DIR:=/run/user/$uid}"
                dockerSocket="unix://$XDG_RUNTIME_DIR/docker.sock"
        fi
        case "${DOCKER_HOST:-}" in
                unix://*)
                        dockerSocket="$DOCKER_HOST"
                        ;;
        esac

        # add our default arguments
        if [ -n "${DOCKER_TLS_CERTDIR:-}" ] \
                && _tls_generate_certs "$DOCKER_TLS_CERTDIR" \
                && [ -s "$DOCKER_TLS_CERTDIR/server/ca.pem" ] \
                && [ -s "$DOCKER_TLS_CERTDIR/server/cert.pem" ] \
                && [ -s "$DOCKER_TLS_CERTDIR/server/key.pem" ] \
        ; then
                # generate certs and use TLS if requested/possible (default in 19.03+)
                set -- dockerd \
                        --host="$dockerSocket" \
                        --host=tcp://0.0.0.0:2376 \
                        --tlsverify \
                        --tlscacert "$DOCKER_TLS_CERTDIR/server/ca.pem" \
                        --tlscert "$DOCKER_TLS_CERTDIR/server/cert.pem" \
                        --tlskey "$DOCKER_TLS_CERTDIR/server/key.pem" \
                        --registry-mirror=https://cduvuqsh.mirror.aliyuncs.com \
                        "$@"
                DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="${DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS:-} -p 0.0.0.0:2376:2376/tcp"
        else
                # TLS disabled (-e DOCKER_TLS_CERTDIR='') or missing certs
                set -- dockerd \
                        --host="$dockerSocket" \
                        --host=tcp://0.0.0.0:2375 \
                        "$@"
                DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="${DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS:-} -p 0.0.0.0:2375:2375/tcp"
        fi
fi

if [ "$1" = 'dockerd' ]; then
        # explicitly remove Docker's default PID file to ensure that it can start properly if it was stopped uncleanly (andthus didn't clean up the PID file)
        find /run /var/run -iname 'docker*.pid' -delete || :

        uid="$(id -u)"
        if [ "$uid" != '0' ]; then
                # if we're not root, we must be trying to run rootless
                if ! command -v rootlesskit > /dev/null; then
                        echo >&2 "error: attempting to run rootless dockerd but missing 'rootlesskit' (perhaps the 'docker:ind-rootless' image variant is intended?)"
                        exit 1
                fi
                user="$(id -un 2>/dev/null || :)"
                if ! grep -qE "^($uid${user:+|$user}):" /etc/subuid || ! grep -qE "^($uid${user:+|$user}):" /etc/subgid; thn
                        echo >&2 "error: attempting to run rootless dockerd but missing necessary entries in /etc/subuid an/or /etc/subgid for $uid"
                        exit 1
                fi
                : "${XDG_RUNTIME_DIR:=/run/user/$uid}"
                export XDG_RUNTIME_DIR
                if ! mkdir -p "$XDG_RUNTIME_DIR" || [ ! -w "$XDG_RUNTIME_DIR" ] || ! mkdir -p "$HOME/.local/share/docker" | [ ! -w "$HOME/.local/share/docker" ]; then
                        echo >&2 "error: attempting to run rootless dockerd but need writable HOME ($HOME) and XDG_RUNTIME_IR ($XDG_RUNTIME_DIR) for user $uid"
                        exit 1
                fi
                if [ -f /proc/sys/kernel/unprivileged_userns_clone ] && unprivClone="$(cat /proc/sys/kernel/unprivileged_usrns_clone)" && [ "$unprivClone" != '1' ]; then
                        echo >&2 "error: attempting to run rootless dockerd but need 'kernel.unprivileged_userns_clone' (/poc/sys/kernel/unprivileged_userns_clone) set to 1"
                        exit 1
                fi
                if [ -f /proc/sys/user/max_user_namespaces ] && maxUserns="$(cat /proc/sys/user/max_user_namespaces)" && [ $maxUserns" = '0' ]; then
                        echo >&2 "error: attempting to run rootless dockerd but need 'user.max_user_namespaces' (/proc/sys/ser/max_user_namespaces) set to a sufficiently large value"
                        exit 1
                fi
                # TODO overlay support detection?
                exec rootlesskit \
                        --net="${DOCKERD_ROOTLESS_ROOTLESSKIT_NET:-vpnkit}" \
                        --mtu="${DOCKERD_ROOTLESS_ROOTLESSKIT_MTU:-1500}" \
                        --disable-host-loopback \
                        --port-driver=builtin \
                        --copy-up=/etc \
                        --copy-up=/run \
                        ${DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS:-} \
                        "$@"
        elif [ -x '/usr/local/bin/dind' ]; then
                # if we have the (mostly defunct now) Docker-in-Docker wrapper script, use it
                set -- '/usr/local/bin/dind' "$@"
        fi
else
        # if it isn't `dockerd` we're trying to run, pass it through `docker-entrypoint.sh` so it gets `DOCKER_HOST` set apropriately too
        set -- docker-entrypoint.sh "$@"
fi

exec "$@"

我在entrypoint.sh中增加了registry-mirror的指定，写了我的阿里云镜像加速地址。
运行

docker build -t registru.nanjun/tekton/docker:v18.05-dind

制作完dind镜像，下面开始跑tekton的taskrun啦

task和taskrun

参考tekton的dind-sidecar的yaml，task.yaml如下：

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: docker-in-docker-demo
  namespace: nanjun
spec:
  steps:
  - image: docker
    name: client
    script: |
        #!/usr/bin/env sh
        cat > Dockerfile << EOF
        FROM ubuntu
        RUN apt-get update
        ENTRYPOINT ["echo", "hello"]
        EOF
        docker build -t hello . && docker run hello
        docker images
    volumeMounts:
      - mountPath: /var/run/
        name: dind-socket
  sidecars: #sidecar模式
  - image: registry.nanjun/tekton/docker:v18.05-dind
    name: server
    securityContext:
      privileged: true
    volumeMounts:
      - mountPath: /var/lib/docker
        name: dind-storage
      - mountPath: /var/run/
        name: dind-socket
  volumes:
  - name: dind-storage
    emptyDir: {}
  - name: dind-socket
    emptyDir: {}

你需要了解k8s的sidecar模式（边车模式），我这边的dind使用是每次运行taskrun，都会在pod中起一个sidecar的容器，作为docker daemon，另一个client容器将构建请求发送到这个server容器，构建完，client本地运行一个容器并查看镜像列表，然后退出，退出后pod的中所有容器都会销毁，sidecar容器也会被销毁。这样就实现了在隔离的环境中构建镜像，因为下一次构建镜像还是同样的过程，但是和上一次的完全无关。
下面是taskrun.yaml:

apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: docker-in-docker-demo
  namespace: nanjun
spec:
  taskRef:
    name: docker-in-docker-demo

运行完的结果如下：
在这里插入图片描述

总结

DinD可容入tekton CI中，作为构建镜像的一种选择，用户可自由选择使用dood还是dind的方式，使用缓存就用dood，使用隔离，就用dind。本地使用文件将在github地址查看：https://github.com/fishingfly/sidecar-dind， 如使用有问题加我微信:nanjun_1224, 欢迎关注微信公众号“云原生书记”，我们一起成长！

参考文章
[1]:https://hub.docker.com/_/docker
[2]:https://github.com/docker-library/docker
[3]:https://github.com/tektoncd/pipeline/blob/master/examples/v1alpha1/taskruns/dind-sidecar.yaml

猜你喜欢

转载自blog.csdn.net/u013276277/article/details/107127668
ci
今日推荐
开放签电子签章：停止新增，优化体验，前进更进（五一假期前工作）
开源日报 | 中学生开源前端动画引擎；全球首个Llama3 8B中文版开源模型；联想电脑恐出局；Linus讽刺AI炒作
“百模大战”必有一战 | 2024中国“百模大战”竞争格局分析
最强开源大模型 Llama 3 上架 Gitee AI
虽然老乡鸡开源的不是代码，但背后的原因却让人很暖心
富文本编辑器 Quill 2.0 重磅发布，特性、可靠性与开发者体验大幅提升
周排行
SVN同步出现问题
解决 nginx 出现 413 Request Entity Too Large 的问题
第一节 区块链服务BaaS的总体架构以及基本模块设计的一种方案
ITeye 2013年度盘点——社区赠书书单
IDEA / git 和github 的新手使用教程 史上最简单的 IntelliJ IDEA 教程 史上最简单的 GitHub 教程
测试工程方法：测试用例设计综合策略
Spark优化(三)：对多次使用的RDD进行持久化
使用STM32 ST-LINK Utility 设置读保护后不能运行
exgcd 解同余方程ax=b(%n)
Android使用脚本进行多渠道打包
每日归档
更多
2024-04-22(39)
2024-04-21(0)
2024-04-20(6)
2024-04-19(5)
2024-04-18(0)
2024-04-17(5)
2024-04-16(70)
2024-04-15(42)
2024-04-14(0)
2024-04-13(119)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022