一、创建PVC
需要先部署NFS
https://blog.51cto.com/yht1990/2630775《storageClass动态挂载对接NFS存储》
kubectl create ns harbor
cat > harbor-pvc.yaml <<'eof'
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: harbor-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
storageClassName: storage-nfs
resources:
requests:
storage: 20Gi
eof
kubectl apply -f harbor-pvc.yaml
二、拉取harbor的chart到本地
[root@k8s-master harbor]# helm repo add harbor https://helm.goharbor.io
[root@k8s-master harbor]# helm repo update
[root@k8s-master harbor]# helm search repo harbor
NAME CHART VERSION APP VERSION DESCRIPTION
harbor/harbor 1.5.1 2.1.1 An open source trusted cloud native registry th...
[root@k8s-master harbor]# helm repo ls
NAME URL
stable http://mirror.azure.cn/kubernetes/charts/
harbor https://helm.goharbor.io
[root@k8s-master harbor]# helm pull harbor/harbor --version 1.5.1
三、Chart参数设置
生产环境size必须调大
[root@k8s-master harbor]# tar xf harbor-1.5.1.tgz
[root@k8s-master harbor]# cd harbor
[root@k8s-master harbor]# cp values.yaml values.yaml.bak
[root@k8s-master harbor]# vim values.yaml
...
36 core: harbor.od.com
...
101 externalURL: https://harbor.od.com # 设置访问域名
...
108 externalURL: https://harbor.od.com
185 193 persistentVolumeClaim:
194 registry:
195 # Use the existing PVC which must be created manually before bound,
196 # and specify the "subPath" if the PVC is shared with other components
197 existingClaim: "harbor-pvc"
198 # Specify the "storageClass" used to provision the volume. Or the default
199 # StorageClass will be used(the default).
200 # Set it to "-" to disable dynamic provisioning
201 storageClass: ""
202 subPath: "registry"
203 accessMode: ReadWriteOnce
204 size: 5Gi
205 chartmuseum:
206 existingClaim: "harbor-pvc"
207 storageClass: ""
208 subPath: "chartmuseum"
209 accessMode: ReadWriteOnce
210 size: 5Gi
211 jobservice:
212 existingClaim: "harbor-pvc"
213 storageClass: ""
214 subPath: "jobservice"
215 accessMode: ReadWriteOnce
216 size: 1Gi
217 # If external database is used, the following settings for database will
218 # be ignored
219 database:
220 existingClaim: "harbor-pvc"
221 storageClass: ""
222 subPath: "database"
223 accessMode: ReadWriteOnce
224 size: 1Gi
225 # If external Redis is used, the following settings for Redis will
226 # be ignored
227 redis:
228 existingClaim: "harbor-pvc"
229 storageClass: ""
230 subPath: "redis"
231 accessMode: ReadWriteOnce
232 size: 1Gi
233 trivy:
234 existingClaim: "harbor-pvc"
235 storageClass: ""
236 subPath: "trivy"
237 accessMode: ReadWriteOnce
238 size: 5Gi
...
539 clair:
540 enabled: false
...
569 trivy:
570 # enabled the flag to enable Trivy scanner
571 enabled: false
...
626 notary:
627 enabled: false
...
四、踩坑一
redis持久化数据目录权限导致无法登录
redis数据目录,/var/lib/redis,需要设置redis的用户及用户组权限
/root/harbor/templates/redis/statefulset.yaml
initContainers:
- name: "change-permission-of-directory"
securityContext:
runAsUser: 0
image: {{ .Values.database.internal.image.repository }}:{{ .Values.database.internal.image.tag }}
imagePullPolicy: {{ .Values.imagePullPolicy }}
command: ["/bin/sh"]
args: ["-c", "chown -R 999:999 /var/lib/redis"]
volumeMounts:
- name: data
mountPath: /var/lib/redis
subPath: {{ $redis.subPath }}
五、踩坑二
registry组件的镜像存储目录权限导致镜像推送失败
registry的镜像存储目录,需要设置registry用户的用户及用户组,不然镜像推送失败
/root/harbor/templates/registry/registry-dpl.yaml
initContainers:
- name: "change-permission-of-directory"
securityContext:
runAsUser: 0
image: {{ .Values.database.internal.image.repository }}:{{ .Values.database.internal.image.tag }}
imagePullPolicy: {{ .Values.imagePullPolicy }}
command: ["/bin/sh"]
args: ["-c", "chown -R 10000:10000 {{ .Values.persistence.imageChartStorage.filesystem.rootdirectory }}"]
volumeMounts:
- name: registry-data
mountPath: {{ .Values.persistence.imageChartStorage.filesystem.rootdirectory }}
subPath: {{ .Values.persistence.persistentVolumeClaim.registry.subPath }}
六、踩坑三
chartmuseum存储目录权限,导致chart推送失败
/root/harbor/templates/chartmuseum/chartmuseum-dpl.yaml
initContainers:
- name: "change-permission-of-directory"
securityContext:
runAsUser: 0
image: {{ .Values.database.internal.image.repository }}:{{ .Values.database.internal.image.tag }}
imagePullPolicy: {{ .Values.imagePullPolicy }}
command: ["/bin/sh"]
args: ["-c", "chown -R 10000:10000 /chart_storage"]
volumeMounts:
- name: chartmuseum-data
mountPath: /chart_storage
subPath: {{ .Values.persistence.persistentVolumeClaim.chartmuseum.subPath }}
七、安装harbor
cd
helm install harbor ./harbor -n harbor
helm -n harbor ls
kubectl -n harbor get po
八、配置访问推送
8.1、域名配置
dns服务器或者hosts里配置
ip harbor.od.com
8.2、配置docker daemon
cat /etc/docker/daemon.json
"insecure-registries": [
"harbor.od.com"
],
systemctl restart docker
8.3 推送chart
使用账户密码登录admin/Harbor12345
docker login harbor.od.com
helm plugin install https://github.com/chartmuseum/helm-push
helm plugin ls
kubectl get secret harbor-harbor-ingress -n harbor -o jsonpath="{.data.ca\.crt}" | base64 -d >harbor.ca.crt
cp harbor.ca.crt /etc/pki/ca-trust/source/anchors
update-ca-trust enable; update-ca-trust extract
helm repo add myharbor https://harbor.od.com/chartrepo/library --ca-file=harbor.ca.crt
helm repo ls
helm push harbor myharbor --ca-file=harbor.ca.crt -u admin -p Harbor12345