Registry搭建实操

一、搭建私有registry

1、registry镜像传递

测试主机ip:192.168.192.225(内网机器)
借助于其他能够访问公网的机器
docker search registry 然后docker save -o ./registry.tar
拷贝到192.168.192.225机器docker load -i registry.tar方式传递registry的docker镜像

打标签:
[root@node1 cert]# docker tag $导入后的rgistry的tag localhost/registry:latest
[root@node1 cert]# mkdir -pv /data/registry/{cert,conf,auth}

2、创建证书

在master1上操作
[root@master1 cert]# vim registry-csr.json

{
  "CN": "registry",
  "hosts": [
      "127.0.0.1",
      "192.168.192.222",
      "192.168.192.223",
      "192.168.192.224",
      "192.168.192.225",
      "192.168.192.226"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "HangZhou",
      "L": "HangZhou",
      "O": "k8s",
      "OU": "FirstOne"
    }
  ]
}

[root@master1 cert]# cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes registry-csr.json | cfssljson -bare registry
拷贝registry*.pem证书到192.168.192.225节点/data/registry/cert 目录下

3、配置文件

[root@node1 registry]# vim /data/registry/conf/config.yml 
[root@node1 registry]# cat conf/config.yml 
version: 0.1
log:
  level: info
  fromatter: text
  fields:
    service: registry

storage:
  filesystem:
    rootdirectory: /var/lib/registry
    maxthreads: 100

http:
  addr: 0.0.0.0:888
  headers:
    X-Content-Type-Options: [nosniff]
  tls:
    certificate: /cert/registry.pem
    key: /cert/registry-key.pem

health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

二、运行测试

1、运行registry

[root@node1 registry]# docker run -itd -p 888:888 --privileged -v /data/registry/data:/var/lib/registry  -v /data/registry/cert:/cert -v /data/registry/conf/config.yml:/etc/docker/registry/config.yml --name registry localhost/registry:latest

2、ca证书分发

[root@master1 docker]# ansible all -i /root/udp/hosts.ini -m shell -a "mkdir /etc/docker/certs.d/192.168.192.225:888/ -pv "
[root@master1 docker]# ansible all -i /root/udp/hosts.ini -m copy -a "src=/etc/kubernetes/cert/ca.pem  dest=/etc/docker/certs.d/192.168.192.225:888/ca.crt"

3、其他节点上可以额正常拉取镜像

[root@master1 service]# ansible all -i /root/udp/hosts.ini -m shell -a "docker pull 192.168.192.225:888/pause:latest   "

4、查看当前有哪些image

[root@node1 conf]# curl -k    https://192.168.192.225:888/v2/_catalog
{"repositories":["addon-resizer","kubernetes-dashboard-amd64","metrics-server-amd64","nginx","pause"]}

三、添加认证

1、修改配置文件

[root@node1 registry]# htpasswd  -Bbn Firstone Passwd123 &> /data/registry/auth/htpasswd
[root@node1 registry]# cat /data/registry/auth/htpasswd
Firstone:$2y$05$0CnJRBMCTYcaL8WNi/2dj.cT3q/RekI2EVo.UUoEEqPb2B2G3vWm6

[root@node1 registry]# cat conf/config.yml 
version: 0.1
log:
  level: info
  fromatter: text
  fields:
    service: registry

storage:
  filesystem:
    rootdirectory: /var/lib/registry
    maxthreads: 100

auth:
  htpasswd:
    realm: basic-realm
    path: /auth/htpasswd

http:
  addr: 0.0.0.0:888
  headers:
    X-Content-Type-Options: [nosniff]
  tls:
    certificate: /cert/registry.pem
    key: /cert/registry-key.pem

health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

2、运行registry

[root@node1 registry]# docker run -itd -p 888:888 --privileged -v /data/registry/data:/var/lib/registry -v /data/registry/auth:/auth  -v /data/registry/cert:/cert -v /data/registry/conf/config.yml:/etc/docker/registry/config.yml --name registry localhost/registry:latest

3、登陆测试

[root@node1 192.168.192.225:888]# docker login 192.168.192.225:888 
Username: Firstone
Password: 
Login Succeeded

登陆成功后的记录信息

[root@node1 192.168.192.225:888]# cat  ~/.docker/config.json
{
    "auths": {
        "127.0.0.1:888": {
            "auth": "Rmlyc3RvbmU6UGFzc3dkMTIz"
        },
        "192.168.192.225:888": {
            "auth": "Rmlyc3RvbmU6UGFzc3dkMTIz"
        }
    }
}

4、上传镜像测试

上传之前需要login，否则会上传失败

[root@node1 192.168.192.225:888]# docker images
REPOSITORY                                       TAG                 IMAGE ID            CREATED             SIZE
192.168.192.225:888/nginx                        latest              719cd2e3ed04        5 weeks ago         109MB
192.168.192.225:888/kubernetes-dashboard-amd64   v1.10.1             f9aed6605b81        7 months ago        122MB
192.168.192.225:888/addon-resizer                1.8.4               5ec630648120        8 months ago        38.3MB
192.168.192.225:888/metrics-server-amd64         v0.3.1              61a0c90da56e        10 months ago       40.8MB
localhost/registry                               latest              265eba1842c4        2 years ago         37.6MB
192.168.192.225:888/pause                        latest              f9d5de079539        5 years ago         240kB
[root@node1 192.168.192.225:888]# for i in $(docker images |awk '{print $1":"$2}') ;do docker  push  $i ;done

5、查询镜像

uri路径:v2/<repoName>/manifests/<tagName> 发 GET 请求
[root@node1 192.168.192.225:888]# curl --user Firstone:Passwd123 --cacert /etc/docker/certs.d/192.168.192.225\:888/ca.crt  https://192.168.192.225:888/v2/nginx/tags/list
{"name":"nginx","tags":["latest"]}
[root@node1 192.168.192.225:888]# curl --user Firstone:Passwd123 --cacert /etc/docker/certs.d/192.168.192.225\:888/ca.crt  https://192.168.192.225:888/v2/addon-resizer/tags/list
{"name":"addon-resizer","tags":["1.8.4"]}

更多API用法参考:https://docs.docker.com/registry/spec/api/

四、daemon.json配置参考

[root@master1 ~]# cat /etc/docker/daemon.json 
{
    "registry-mirrors": ["192.168.192.225:888"],
    "max-concurrent-downloads": 20,
    "live-restore": true,
    "max-concurrent-uploads": 10,
    "debug": true,
    "log-opts": {
      "max-size": "100m",
      "max-file": "5"
    }
}

原理介绍：

加密传输：对称加密和非对称加密 //实际使用的是对称加密传输

对称加密：解密和加密使用的是同一个秘钥，不安全。因为在协商秘钥的过程中使用的是明文传输

非对称加密：私钥加密公钥解密或者公钥加密私钥解密

协商秘钥过程：为了安全，使用非对称加密，用对方的公钥加密后传输给对方 //非对称加密算法进行对称加密算法协商过程

安全的获取公钥：CA出现了，使用数字证书签发机构颁发的证书来保证非对称加密过程本身的安全
1)client->访问server，server把自己的证书返回给client(证书包含证书的颁发机构、有效期、公钥、证书持有者、签名等)
2)client去查找操作系统中已内置的受信任的证书发布机构CA与服务器发来的证书中的颁发者CA比对，用于校验证书是否为合法机构颁发
3)找不到就认为不可行，找到了client从操作系统中取出颁发者CA 的公钥，然后对服务器发来的证书里面的签名进行解密
使用相同的hash算法计算出服务器发来的证书的hash值，将这个计算的hash值与证书中签名做对比，结果一致就是合法
4)clent 读取证书中的公钥，用于后续加密了

问题记录：

1、清理之前的registry的时候报错
[root@node1 ~]# docker rm 6f0d1bcd9f87
Error response from daemon: driver "overlay" failed to remove root filesystem for 6f0d1bcd9f87a62f9b991d18d460c215f49633d16559bb07eca2ed3d1c1742fd: remove /var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/merged: device or resource busy
[root@node1 ~]# grep docker /proc/*/mountinfo | grep ec8a0744de1
/proc/20276/mountinfo:125 110 0:37 / /var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/merged rw,relatime shared:60 - overlay overlay rw,lowerdir=/var/lib/docker/overlay/59fce193b8b2ab730f7c4c556d2ac931c1567e772efb72aafcb29716287bffc2/root,upperdir=/var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/upper,workdir=/var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/work
[root@node1 ~]# ps -ef |grep 20276
root 19972 18147 0 14:22 pts/0 00:00:00 grep --color=auto 20276
ntp 20276 1 0 Jul18 ? 00:00:00 /usr/sbin/ntpd -u ntp:ntp -g
[root@node1 ~]# service ntpd restart
[root@node1 ~]# docker rm 6f0d1bcd9f87

2、拉取镜像报错certificate signed by unknown authority
解法1：docker.service ExecStart=/usr/bin/dockerd --insecure-registry 镜像所在的地址
解法2：[root@node1 192.168.192.234:888]# ls
/etc/docker/certs.d/192.168.192.234:888/ca.pem
[root@node1 192.168.192.234:888]# mv ca.pem ca.crt

备注：在安装过程中，可以只开启https即可

参考文档：
https://docs.docker.com/registry/deploying/
https://docs.docker.com/registry/configuration/#list-of-configuration-options
https://deepzz.com/post/secure-docker-registry.html
https://blog.51cto.com/11883699/2160032