以statefulset方式在k8s集群中部署zookeeper3.6.3集群（二）

上一节有讲到制作zookeeper3.6.3的docker镜像，本节将来展开在k8s集群中部署zookeeper3.6.3集群。

背景

目前我司已经将java程序以微服务的形式迁移到k8s环境，但是由于之间在ECS上运行的zk集群版本是3.6.3，所以需要在容器中使用相应的版本且需要数据持久化，容许zk服务在重启后数据仍在。

需求分析

statefulset是k8s中专门为解决有状态服务而生的一种资源，可以保证容器在重启后可以有序的启动且数据不会丢失，所以已下通过statefulset的方式在k8s中部署zk集群。

环境说明

1 本文的操作环境为 Kubernetes v1.18.0 版本，使用docker的容器运行时，docker 和 Kubelet 采用的 cgroup 驱动为 systemd。

2 采用的是动态存储的方式持久化zk集群的数据

准备yaml文件

step1：编写zk的配置文件

cat <<EOF> zk-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: zk-cm
data:
  jvm.heap: "1G"
  tick: "2000"
  init: "10"
  sync: "5"
  client.cnxns: "1000"
  snap.retain: "3"
  purge.interval: "0"

step2：编写zk的pdb文件，保证集群最少可用的节点数

cat <<EOF> zk-pdb.yaml
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: zk-pdb
spec:
  selector:
    matchLabels:
      app: zk
  minAvailable: 2

step3：编写zk的svc文件

cat <<EOF>zk-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: zk-cs
  labels:
    app: zk
spec:
  selector:
    app: zk
  ports:
  - port: 2181
    name: client
---
apiVersion: v1
kind: Service
metadata:
  name: zk-svc
  labels:
    app: zk-svc
spec:
  ports:
  - port: 2888
    name: server
  - port: 3888
    name: leader-election
  clusterIP: None
  selector:
    app: zk

step4：编写zk的sts文件

cat <<EOF> zk-sts.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zk
spec:
  selector:
    matchLabels:
      app: zk
  serviceName: zk-svc
  replicas: 3
  template:
    metadata:
      labels:
        app: zk
    spec:
      imagePullSecrets:
      - name: jenkins-admin-token
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values: 
                    - zk
              topologyKey: "kubernetes.io/hostname"
      containers:
      - name: k8szk
        imagePullPolicy: Always
        image: registry.cn-hangzhou.aliyuncs.com/my_image_repo/zookeeper:3.6.3
        resources:
          requests:
            memory: "2Gi"
            cpu: "500m"
        ports:
        - containerPort: 2181
          name: client
        - containerPort: 2888
          name: server
        - containerPort: 3888
          name: leader-election
        env:
        - name : ZK_REPLICAS
          value: "3"
        - name : ZK_HEAP_SIZE
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: jvm.heap
        - name : ZK_TICK_TIME
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: tick
        - name : ZK_INIT_LIMIT
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: init
        - name : ZK_SYNC_LIMIT
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: tick
        - name : ZK_MAX_CLIENT_CNXNS
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: client.cnxns
        - name: ZK_SNAP_RETAIN_COUNT
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: snap.retain
        - name: ZK_PURGE_INTERVAL
          valueFrom:
            configMapKeyRef:
                name: zk-cm
                key: purge.interval
        - name: ZK_CLIENT_PORT
          value: "2181"
        - name: ZK_SERVER_PORT
          value: "2888"
        - name: ZK_ELECTION_PORT
          value: "3888"
        command:
        - sh
        - -c
        - zkGenConfig.sh && zkServer.sh start-foreground
        readinessProbe:
          exec:
            command:
            - "zkOk.sh"
          initialDelaySeconds: 10
          timeoutSeconds: 5
        livenessProbe:
          exec:
            command:
            - "zkOk.sh"
          initialDelaySeconds: 10
          timeoutSeconds: 5
        volumeMounts:
        - name: datadir
          mountPath: /data/zookeeper
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: datadir
      annotations:
        volume.beta.kubernetes.io/storage-class: "glusterfs"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 10G

step5：依次执行yaml文件

kubectl apply -f zk-cm.yaml
kubectl apply -f zk-pdb.yaml
kubectl apply -f zk-svc.yaml
kubectl apply -f zk-sts.yaml

step6：验证

zk集群个节点的状态

zk的版本

zk数据的持久化

zk服务的正常性

至此完整zk集群在k8s中的部署，都到这儿了，更多文章，详见个人微信公众号ALL In Linux，来扫一扫吧！