Benpian has joined " the .NET Core ON K8S study and practice series index ", you can click to see more container technology related series of articles.

In Docker we know, in order to achieve persistent data (so-called Docker data persistence that is, data is not ended with the end of the Container ), the need for data from the host to mount a container, commonly used means is Volume data volumes . In K8S, but also provides a storage model Volume, we will support data applications in the persistent store into the container.

A, Volume

About 1.1 K8S Volume

　　In order to save persistent data containers, we can use K8S Volume, which essentially is a table of contents, is no different Docker Volume.

　　Note that: K8S Volume lifecycle independent of the vessel, Pod in the container may be destroyed and rebuilt, but Volume is retained.

　　When Volume Pod is to mount, all of the containers Pod can access the Volume. In K8S, the backend supports a variety of types, such as emptyDir, hostPath, NFS, Ceph storage service, etc. as well as some of the cloud service provider. For Pod, it does not need to be concerned about in the end the data will be stored in the local file system or remote Drive, it considers all types of storage are just a Volume directory only.

1.2 K8S Volume

　　(1) emptyDir

　　As K8S the most basic type of Volume, emptyDir provides the most basic persistence scheme, but this program is not very good. Because, emptyDir for Pod is not persistent (it is the persistence of the container is), because when a node is removed from the Pod, Volume content will be deleted. But only if the container is destroyed while still Pod, the Volume will not be affected.

　　In other words: emptyDir Volume lifecycle consistent with the Pod . In view of this feature is not recommended to use this type Volume in practice.

　　（2）hostPath

　　Relative to emptyDir, hotPath overcomes its weaknesses life cycle, if the Pod is destroyed, hostPath corresponding directory will still be retained. However, if once the Host crashes, hostPath it can not be visited. Because, hostPath is the Docker Host file system directory that already exists to mount Pod container, it will depend on the Host.

　　In K8S, those who need to access the internal data K8S or Docker (binaries and configuration files) to the application requires hostPath, such kube-apiserver and applications such kube-controller-manager. The following configuration is kube-apiserver persistence setting that defines three hostPath: ca-certs, etc-pki and k8s-certs, respectively Host directory / etc / ssl / certs, / etc / pki and / etc / kubernetes / pki.

    volumeMounts:
    - mountPath: /etc/ssl/certs
      name: ca-certs
      readOnly: true
    - mountPath: /etc/pki
      name: etc-pki
      readOnly: true
    - mountPath: /etc/kubernetes/pki
      name: k8s-certs
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  hostNetwork: true
  nodeName: k8s-master
  priority: 2000000000
  nodeName: k8s-master
  priority: 2000000000
  priorityClassName: system-cluster-critical
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    operator: Exists
  volumes:
  - hostPath:
      path: /etc/ssl/certs
      type: DirectoryOrCreate
    name: ca-certs
  - hostPath:
      path: /etc/pki
      type: DirectoryOrCreate
    name: etc-pki
  - hostPath:
      path: /etc/kubernetes/pki
      type: DirectoryOrCreate
    name: k8s-certs

　　(3) External Storage Povider

　　If our K8S are deployed on AWS, GCE, Azure and other public cloud, you can use the hard drive as a direct cloud Volume. Since I do not use here, so skip with friends can refer to the use of direct reference documentation for each cloud service provider's configuration.

二、PersistentVolume与PersistentVolumeClaim

About 2.1 PersistentVolume and PersistentVolumeClaim

　　Several programs mentioned earlier are deficiencies in manageability, especially when large-scale cluster time, efficiency and safety are to be improved. Therefore, K8S offers a solution: PersistentVolume and PersistentVolumeClaim , hereinafter referred to as PV and PVC.

　　PV is a storage space of external storage systems, created and maintained by the administrator. As with Volume, PV persistent, life-cycle independent of the Pod.

　　PVC is the application of the PV (Claim), PVC is usually created and maintained by ordinary users. When you need to allocate storage resources Pod, the user can create a PVC, size and access method specified capacity storage resources (such as ReadOnly) and other information, K8S will find and offer to meet the conditions of PV.

　　Learn ASP.NET Identity of children's shoes should not be unfamiliar to Claim word, if we authentication information as a Claims, then one of a key-value pair is the Claim. We used ClaimTypes shown below, we can locate the authentication information by Value Claim.

　　Similarly, we know that the Claim will be positioned to address the PV which we want to use.

　　Like with K8S Volume, K8S PersistentVolume also supports multiple types of storage, such as NFS, AWS EBS, Ceph and so on.

2.2 NFS PV use

　　NFS is the Network File System (Network File System), which allows a system to share local directories and files to other systems on the network. By NFS, users and applications can access files on remote systems as if they were local files.

　　About how to configure NFS as CentOS, please refer to this article " CentOS7 install NFS service ."

　　It is assumed that an NFS server has been set up for our k8s-master node, directory / edc / k8s / nfsdata, as shown below:

　　(1) create a PV

　　 Next we have to create a PV, which yaml configuration file as follows:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: edc-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: nfs
  nfs:
    path: /edc/k8s/nfsdata/edc-pv
    server: 192.168.2.100

　　其中：

capacity指定了PV的容量为1GB
accessModes指定访问模式为ReadWriteOnce，表示PV能够以Read-Write模式mount到单个节点。此外，还支持ReadOnlyMany和ReadWriteMany，分别代表PV能以Read-Only模式或者Read-Write模式mount到多个节点。这里ReadWriteOnce只mount到单个节点，即k8s-master（192.168.2.100）。
persistentVolumeReclaimPolicy指定了此PV的回收策略为Recycle，表示清除PV中的数据。此外，还支持Retain和Delete，Retain表示需要管理员手动回收，类似于你用C/C++还需要手动写free代码释放空间。而Delete呢，表示删除Storage Provider中的对应存储资源，如果你使用的是外部云服务提供商的存储空间的话。
storageClassName指定了PV的class为nfs。
nfs配置项指定了PV在NFS服务器上对应的目录，如果没有可以事先创建一下。

　　理解了其中的配置项，我们创建该PV，可以看到其状态Status变为了Available，表示可以被PVC申请啦。

　　（2）创建一个PVC

　　与创建PV不同，创建PVC只需指定PV容量、访问模式以及class即可：

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: edc-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: nfs

　　有了配置文件，就可以创建PVC了：

　　可以看到，edc-pvc已经Bound到edc-pv了，申请PV成功。

　　申请成功之后，我们就可以在Pod中使用了，下面是一个示例Pod的配置文件：

apiVersion: v1
kind: Pod
metadata:
  name: edc-pv-pod
spec:
  containers:
  - name: edc-pv-pod
    image: busybox
    args:
    - /bin/sh
    - -c
    - sleep 30000
    volumeMounts:
    - mountPath: "/mydata"
      name: mydata
  volumes:
    - name: mydata
      persistentVolumeClaim:
        claimName: edc-pvc

　　通过kubectl创建该pod，如下所示：

　　接下来验证一下PV是否可用：

　　可以看到，在Pod中创建的文件/mydata/hello已经保存到了NFS服务器目录的edc-pv目录下了。

2.2 NFS PV的回收

　　当我们不再需要某个PV时，也可以使用PVC来回收PV，如下所示：

kubectl delete pvc edc-pvc

　　当edc-pvc被删除后，我们会发现K8S启动了一个新Pod，这个Pod就是用来清除edc-pv的数据的。数据的清理需要一个过程，完成后edc-pv的状态会重新恢复为Available，此时可以被新的PVC申请。

　　此外，由于我们设置的回收策略为Recycle，所以Pod中的数据也被清除了：

　　如果希望能够保留这些数据，那么我们需要将回收策略改为Retain：

apiVersion: v1
kind: PersistentVolume
metadata:
  name: edc-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nfs
  nfs:
    path: /edc/k8s/nfsdata/edc-pv
    server: 192.168.2.100

　　这里就不再验证Retain的效果了。

三、MySQL持久化存储案例

3.1 准备工作

　　这里我们来演示一个MySQL持久化存储的案例：

　　（1）创建PV和PVC

　　准备PV和PVC的yaml：

-- mysql-pv
apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nfs
  nfs:
    path: /edc/k8s/nfsdata/mysql-pv
    server: k8s-master

-- mysql-pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: nfs

　　通过kubectl apply创建PV和PVC：

kubectl apply -f mysql-pv.yaml
kubectl apply -f mysql-pvc.yaml

　　可以看到，mysql-pvc已经申请到了mysql-pv。

　　（2）部署MySQL

　　准备yaml配置文件：

apiVersion: v1
kind: Service
metadata:
  name: mysql-service
spec:
  ports:
  - port: 3306
  selector:
    app: mysql

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: password
        ports:
        - containerPort: 3306
          name: mysql-container
        volumeMounts:
        - name: mysql-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-storage
        persistentVolumeClaim:
          claimName: mysql-pvc

　　重点关注其中的volumeMounts和volumes配置，其中mysql-pvc申请Bound的mysql-pv将会被mount到MySQL的数据目录/var/lib/mysql下。

　　通过kubectl创建MySQL：　　

kubectl apply -f mysql-service.yaml
kubectl get pod -o wide

　　可以看到，MySQL被部署到了k8s-node1节点上。

　　（3）客户端访问MySQL

　　下面我们在k8s-master上通过客户端访问MySQL Service：

kubectl run -it --rm --image=mysql:5.6 --restart=Never mysql-client -- mysql -h mysql-service -ppassword

　　如下图所示，进入了MySQL数据库：

　　接下来我们更新一下数据库，如下图所示：

　　新建了一张表edc_test，插入了一行数据1110.

3.2 快速验证

　　（1）模拟k8s-node1故障

　　接下来我们模拟一下k8s-node1宕机，这样在k8s-node1上运行的MySQL服务就会受到影响，不过根据之前的了解，K8S会帮我们将MySQL迁移到k8s-node2上从而保证服务可用。

　　首先，关闭k8s-node1：

shutdown now

　　其次，验证K8S迁移MySQL：

　　（2）验证数据一致性

　　虽然k8s-node1挂了，但是K8S帮我们迁移了MySQL到k8s-node2，而且数据也是完好无损，如下图所示：

　　（3）验证数据持久性

　　如果我们将部署的Service和Deployment删掉，那么其Pod也会停止被删除，但是由于我们的PV的回收策略是Retain，因此其数据不会被清除：

四、小结

　　本文探索了K8S的数据管理方案Volume，其中普通类型的Volume如emptyDir和hostPath虽然使用方便，但是可持久性不强，而外部云存储Volume Provider则提供了更好的持久化存储。PV和PVC的模式，更加适合于我们使用在实际环境中，最后还通过了一个MySQL持久化案例演示了如何应用PV和PVC实现持久化。