利用NFS动态提供Kubernetes后端存储卷

发表于   |   更新于 | 分类于 | | 阅读次数: | 总览

        nfs-client-provisioner是一个automatic provisioner，使用NFS作为存储，自动创建PV和对应的PVC，本身不提供NFS存储，需要外部先有一套NFS存储服务。

  • PV以 ${namespace}-${pvcName}-${pvName}的命名格式提供（在NFS服务器上）
  • PV回收的时候以 archieved-${namespace}-${pvcName}-${pvName} 的命名格式（在NFS服务器上）

官方访问地址

1、权限体系构建

1.1、创建serviceaccount

ServiceAccount也是一种账号, 供运行在pod中的进程使用, 为pod中的进程提供必要的身份证明

1
2
3
4
5
6
7
$ cat > serviceaccount.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: kube-ops
EOF

1.2、创建role

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ cat >clusterrole.yaml<<EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
  namespace: kube-ops
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["services", "endpoints"]
    verbs: ["get", "create","list", "watch","update"]
  - apiGroups: ["extensions"]
    resources: ["podsecuritypolicies"]
    resourceNames: ["nfs-client-provisioner"]
    verbs: ["use"]
EOF

1.3、账户和角色绑定

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ cat >clusterrolebinding.yaml <<EOF
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: kube-ops
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: kube-ops
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
EOF

1.4、执行创建

1
2
3
4
$ kubectl create -f serviceaccount.yaml -f clusterrole.yaml -f clusterrolebinding.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created

2、安装部署

        下载deployment.yaml文件,需要修改NFS服务器所在的IP地址（10.10.10.60），以及NFS服务器共享的路径（/ifs/kubernetes），两处都需要修改为你实际的NFS服务器和共享目录

2.1、部署存储供应卷

根据PVC的请求, 动态创建PV存储.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
$ cat > deployment.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: kube-ops
  name: nfs-client-provisioner
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  namespace: kube-ops
  name: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 172.21.17.39
            - name: NFS_PATH
              value: /opt
      volumes:
        - name: nfs-client-root
          nfs:
            server: 172.21.17.39
            path: /opt
EOF
  • 修改StorageClass文件并部署class.yaml

此处可以不修改，或者修改provisioner的名字，需要与上面的deployment的PROVISIONER_NAME名字一致

2.2、创建storageclass

1
2
3
4
5
6
7
8
9
$ cat > class.yaml <<EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"
EOF

2.3、执行创建

1
2
3
4
5
6
$ kubectl apply -f deployment.yaml 
serviceaccount/nfs-client-provisioner created
deployment.extensions/nfs-client-provisioner created

$ kubectl apply -f class.yaml 
storageclass.storage.k8s.io/managed-nfs-storage created
2.3.1、查看StorageClass
1
2
3
$ kubectl get storageclass
NAME                  PROVISIONER      AGE
managed-nfs-storage   fuseim.pri/ifs   18s
2.3.2、设置默认后端存储

设置这个default名字的SC为Kubernetes的默认存储后端

1
2
kubectl patch storageclass managed-nfs-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io/managed-nfs-storage patched
  • storage.yaml (和上面一样)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    $ cat > storage.yaml <<EOF
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: managed-nfs-storage
      annotations:
        storageclass.kubernetes.io/is-default-class: "true"
    provisioner: fuseim.pri/ifs
    parameters:
      archiveOnDelete: "false"
    EOF

2.3.3、查看验证

1
2
3
4
5
6
7
8
9
$ kubectl get all -n kube-ops
NAME                                          READY   STATUS    RESTARTS   AGE
pod/nfs-client-provisioner-77f678858b-8d2d6   1/1     Running   0          26m

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nfs-client-provisioner   1/1     1            1           29m

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/nfs-client-provisioner-77f678858b   1         1         1       26m

3、验证测试

3.1、创建一个测试存储

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ cat > test-claim.yaml <<EOF
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
  namespace: kube-ops
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi
EOF

3.2、启动测试POD

POD文件如下，作用就是在test-claim的PV里touch一个SUCCESS文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ cat  > test-pod.yaml <<EOF
kind: Pod
apiVersion: v1
metadata:
  name: test-pod
  namespace: kube-ops
spec:
  containers:
  - name: test-pod
    image: docker.io/busybox:1.24
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: test-claim
EOF

3.3、执行创建

1
2
3
$ kubectl apply -f ./
persistentvolumeclaim/test-claim created
pod/test-pod created

3.4、查看验证

1
2
3
4
5
6
7
$ kubectl get pod,pv -n kube-ops
NAME                                          READY   STATUS      RESTARTS   AGE
pod/nfs-client-provisioner-77f678858b-8d2d6   1/1     Running     0          3h26m
pod/test-pod                                  0/1     Completed   0          172m

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                 STORAGECLASS          REASON   AGE
persistentvolume/pvc-2f0057b0-df35-11e9-ad62-fa163e53d4c8   1Mi        RWX            Retain           Bound    kube-ops/test-claim   managed-nfs-storage            172m
  • 登录nfs服务器查看是否成功的创建目录
    1
    2
    $ ls /opt/
    kube-ops-test-claim-pvc-2f0057b0-df35-11e9-ad62-fa163e53d4c8

3.5、更改PersistentVolumes 中的一个回收策略

  • 查看集群中PersistentVolumes

    1
    2
    3
    $ kubectl get pv -n kube-ops
    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                 STORAGECLASS          REASON   AGE
    pvc-2f0057b0-df35-11e9-ad62-fa163e53d4c8   1Mi        RWX            Delete           Bound    kube-ops/test-claim   managed-nfs-storage            3m6s

  • 更改PersistentVolumes

    1
    2
    3
    4
    5
    6
    $ kubectl patch pv pvc-2f0057b0-df35-11e9-ad62-fa163e53d4c8  -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
    persistentvolume/pvc-2f0057b0-df35-11e9-ad62-fa163e53d4c8 patched

    $ kubectl get pv -n kube-ops
    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                 STORAGECLASS          REASON   AGE
    pvc-2f0057b0-df35-11e9-ad62-fa163e53d4c8   1Mi        RWX            Retain           Bound    kube-ops/test-claim   managed-nfs-storage            3m54s

  • 删除测试

    1
    2
    # kubectl delete -f test-pod.yaml 
    # kubectl delete -f test-claim.yaml
坚持原创技术分享，您的支持将鼓励我继续创作！
0%