nfs为kubernetes动态提供后端卷

本文将介绍使用nfs-client-provisioner，把NFS Server给Kubernetes作为持久存储的后端，并且动态提供PV。

nfs-client-provisioner 是一个Kubernetes的简易NFS的外部provisioner，本身不提供NFS，需要现有的NFS服务器提供存储。

• PV以 ${namespace}-${pvcName}-${pvName}的命名格式提供（在NFS服务器上）
• PV回收的时候以 archieved-${namespace}-${pvName} 的命名格式（在NFS服务器上）
  确认nfs server 以及存在（此处不进行nfs的配置，具体可以参考网络资料）

一、配置rbac

nfs provision使用单独的serviceaccount账户，yaml内容如下：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: default
  
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nfs-client-provisioner     ###我创建的权限给了最大。
  namespace: default
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - get
  - list
  - watch
  - create
  - patch
  - update
  - exec
  - delete

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nfs-client-provisioner
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nfs-client-provisioner
subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  namespace: default


[root@master ~]# kubectl get ClusterRoleBinding   | grep nfs
nfs-client-provisioner    ClusterRole/nfs-client-provisioner       4d6h
[root@master ~]# kubectl get ClusterRole   | grep nfs
nfs-client-provisioner                                             2023-03-23T07:40:14Z
[root@master ~]# kubectl get sa  | grep nfs
nfs-client-provisioner   0         4d6h
[root@master ~]# 

二、启动nfs provisioner

注意使用的镜像必须为最新，否则为出现（unexpected error getting claim reference: selfLink was empty, can’t make reference这个错误，之前网上查阅资料在api里面开启--feature-gates=RemoveSelfLink=false这个参数，但是在1.20版本之后此参数已经弃用，1.24版本之后添加此参数会造成api无法启动问题。）

kind: Deployment
apiVersion: apps/v1 
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner   ###上面创建的sa的名称
      containers:
        - name: nfs-client-provisioner
          image: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0   ####注意使用tag 为latest的镜像会出现上述的问题。
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs        ###provisioner 名称
            - name: NFS_SERVER   ###nfs的server地址
              value: 192.168.5.240
            - name: NFS_PATH    ###nfs挂载的路径
              value: /data
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.5.240    ###同上
            path: /data
[root@master ~]# 

[root@master ~]# kubectl get pod 
NAME                                      READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-6fb6f65f8b-4sxgp   1/1     Running   0          4d6h

三、创建storageclass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs    ####上述定义的provisioner 名称


[root@master ~]# kubectl get sc 
NAME                  PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   fuseim.pri/ifs   Delete          Immediate           false                  4d6h

四、创建pvc测试

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: managed-nfs-storage      ###sc name
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi    ###pvc大小
      
      
[root@master ~]# kubectl get pvc | grep test 
test-claim              Bound    pvc-82480978-dff0-4b82-bd15-dcc14d0617de   1Gi        RWX            managed-nfs-storage   48s
[root@master ~]# 
[root@master ~]# 

[root@master ~]# kubectl get pv | grep pvc-82480978-dff0-4b82-bd15-dcc14d0617de
pvc-82480978-dff0-4b82-bd15-dcc14d0617de   1Gi        RWX            Delete           Bound    default/test-claim                 managed-nfs-storage            67s

五、删除pvc

[root@master ~]# kubectl delete pvc test-claim 
persistentvolumeclaim "test-claim" deleted
[root@master ~]# 
[root@master ~]# kubectl get pvc | grep test 
[root@master ~]# kubectl get pv | grep pvc-82480978-dff0-4b82-bd15-dcc14d0617de


查看nfs 路径下的pvc，如下：
[root@master data]# ll  | grep pvc-82480978-dff0-4b82-bd15-dcc14d0617de
drwxrwxrwx  2 root root  4096 Mar 27 22:30 archived-pvc-82480978-dff0-4b82-bd15-dcc14d0617de