View All Products
Compute
Storage
Databases
Networking
Developer Tools
Delivery
Security
Services
Industries
Pricing
Community
Engage With Us

Setting Up Rook NFS for Persistent Storage on LKE

Published by Todd Becker
Traducciones al Español
Estamos traduciendo nuestros guías y tutoriales al Español. Es posible que usted esté viendo una traducción generada automáticamente. Estamos trabajando con traductores profesionales para verificar las traducciones de nuestro sitio web. Este proyecto es un trabajo en curso.
Deprecated

This guide has been deprecated and is no longer being maintained.

Rook NFS allows remote hosts to mount filesystems over a network and interact with those filesystems as though they are mounted locally. When used with LKE, Rook can mount a Linode Block Storage PVC which uses ReadWriteOnce permissions. The volume can then be leveraged as NFS and exported as a storage class that uses ReadWriteMany permissions. This allows Linode’s Block Storage to store persistent data for LKE clusters.

Before you Begin

  • This guide assumes that you already have an LKE cluster up and running. If that is not the case, please follow the instructions in our LKE Tutorial.
  • This guide relies on git wherever kubectl is installed. While git is installed on many Linux distributions, others may require manual installation. Git can be installed on most distributions by following our Git Installation Guide

Installing Rook NFS on LKE

  1. Rook has several manifests on their github repository that are used within this guide. Clone the latest release from the project’s repository to your machine:

    git clone --single-branch --branch v1.7.3 https://github.com/rook/nfs.git
    Note
    If you do not want to use git, the raw manifest files can alternatively be obtained via wget or curl.

  2. Apply the Rook operator and common manifests:

    kubectl create -f ~/nfs/cluster/examples/kubernetes/nfs/crds.yaml
kubectl create -f ~/nfs/cluster/examples/kubernetes/nfs/operator.yaml

  3. Validate that all pods are running as expected:

    kubectl -n rook-nfs-system get pod

    You Should see the following output:

    NAME                                 READY   STATUS    RESTARTS   AGE
rook-nfs-operator-5cc679885d-88769   1/1     Running   0          45h

  4. Overwrite the contents of the webhook.yaml file to the contents of the following file. This ensures that the webhook manifests will rely on a more up to date version of the Kubernetes API, ensuring that they’re compatible with the latest versions of Kubernetes:

    File: /nfs/cluster/examples/kubernetes/nfs/webhook.yaml
      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

    apiVersion: v1
kind: ServiceAccount
metadata:
  name: rook-nfs-webhook
  namespace: rook-nfs-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: rook-nfs-webhook
  namespace: rook-nfs-system
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames:
      - "rook-nfs-webhook-cert"
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: rook-nfs-webhook
  namespace: rook-nfs-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: rook-nfs-webhook
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: rook-nfs-webhook
    namespace: rook-nfs-system
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: rook-nfs-webhook-cert
  namespace: rook-nfs-system
spec:
  dnsNames:
    - rook-nfs-webhook.rook-nfs-system.svc
    - rook-nfs-webhook.rook-nfs-system.svc.cluster.local
  issuerRef:
    kind: Issuer
    name: rook-nfs-selfsigned-issuer
  secretName: rook-nfs-webhook-cert
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: rook-nfs-selfsigned-issuer
  namespace: rook-nfs-system
spec:
  selfSigned: {}
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  annotations:
    cert-manager.io/inject-ca-from: rook-nfs-system/rook-nfs-webhook-cert
  creationTimestamp: null
  name: rook-nfs-validating-webhook-configuration
webhooks:
  - clientConfig:
      caBundle: Cg==
      service:
        name: rook-nfs-webhook
        namespace: rook-nfs-system
        path: /validate-nfs-rook-io-v1alpha1-nfsserver
    failurePolicy: Fail
    admissionReviewVersions: ["v1", "v1beta1"]
    sideEffects: None
    name: validation.nfsserver.nfs.rook.io
    rules:
      - apiGroups:
          - nfs.rook.io
        apiVersions:
          - v1alpha1
        operations:
          - CREATE
          - UPDATE
        resources:
          - nfsservers
---
kind: Service
apiVersion: v1
metadata:
  name: rook-nfs-webhook
  namespace: rook-nfs-system
spec:
  selector:
    app: rook-nfs-webhook
  ports:
    - port: 443
      targetPort: webhook-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rook-nfs-webhook
  namespace: rook-nfs-system
  labels:
    app: rook-nfs-webhook
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rook-nfs-webhook
  template:
    metadata:
      labels:
        app: rook-nfs-webhook
    spec:
      containers:
        - name: rook-nfs-webhook
          image: rook/nfs:v1.7.3
          imagePullPolicy: IfNotPresent
          args: ["nfs", "webhook"]
          ports:
            - containerPort: 9443
              name: webhook-server
          volumeMounts:
            - mountPath: /tmp/k8s-webhook-server/serving-certs
              name: cert
              readOnly: true
      volumes:
        - name: cert
          secret:
            defaultMode: 420
            secretName: rook-nfs-webhook-cert

  5. Apply the Rook Admission Webhook and Cert Manager. Cert manager is a prerequisite for the webhook and applies an added layer of security:

    kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.8.0/cert-manager.yaml
kubectl apply -f  ~/nfs/cluster/examples/kubernetes/nfs/webhook.yaml

  6. Verify that the webhook and cert manager were set up correctly:

    kubectl get -n cert-manager pod
kubectl -n rook-nfs-system get pod

    Output similar to the following should appear:

    kubectl get -n cert-manager pod
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-cainjector-6d9776489b-wprdx   1/1     Running   0          45h
cert-manager-d7d8fb5c9-wv66w               1/1     Running   0          45h
cert-manager-webhook-544c44ccdc-stjjb      1/1     Running   0          45h
kubectl -n rook-nfs-system get pod
NAME                                 READY   STATUS    RESTARTS   AGE
rook-nfs-operator-5cc679885d-88769   1/1     Running   0          45h
rook-nfs-webhook-6ffb579d8c-wl59k    1/1     Running   0          45h

  7. There is an optional step in the Rook guide to add a Pod Security Policy. This is recommended for security.

    kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/psp.yaml

  8. Create a service account user for Rook to run the NFS server with:

    kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/rbac.yaml

  9. Edit the PVC portion of the default NFS server manifests in the nfs.yaml file before initializing the NFS server. The provided NFS manifest has two changes that need to be made. The first is the storage class for the PVC is left off, which assumes the default storage class. This can be explicitly defined to “linode-block-storage-retain” instead of assuming the default storage class. Secondly the accessModes is set to ReadWriteMany and the Linode block storage does not support ReadWriteMany. This should be changed to ReadWriteOnce.

    File: /nfs/cluster/examples/kubernetes/nfs/nfs.yaml
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

    ---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-default-claim
  namespace: rook-nfs
spec:
  storageClassName: linode-block-storage-retain # Add this line to specify the storage class to be used
  accessModes:
  - ReadWriteOnce # Edit this line to ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
# Additionally this manifest contains the Rook NFSServer below the PVC
# This is not displayed as there are no modifications to that portion of the manifest

  10. Apply the updated nfs.yaml file, then add a Rook storage class that leverages the Rook NFS server.

    kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/nfs.yaml
kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/sc.yaml

Testing the Server

To test the Rook NFS server, create two deployments that will leverage a singular storage class labeled as rook-nfs-share1.

  1. Deploy the PVC, busybox, and web-rc server using the following commands:

    kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/pvc.yaml
kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/busybox-rc.yaml
kubectl apply -f ~/nfs/cluster/examples/kubernetes/nfs/web-rc.yaml

  2. Create a service that will allow the busybox container to update the timestamp in the index.html file:

    kubectl apply -f  ~/nfs/cluster/examples/kubernetes/nfs/web-service.yaml

The following command can now be entered to list the files being stored for in the volume for the busybox workload:

kubectl exec $(kubectl get pod -l app=nfs-demo,role=busybox -o jsonpath='{.items[0].metadata.name}') -- ls /mnt

This additional command can also be entered to list all of the files in the volume for the nginx workload:

kubectl exec $(kubectl get pod -l app=nfs-demo,role=web-frontend -o jsonpath='{.items[0].metadata.name}') -- ls /usr/share/nginx/html

This page was originally published on

docker kubernetes container nginx linode platform

Your Feedback Is Important

Let us know if this guide was helpful to you.

Join the conversation.
Read other comments or post your own below. Comments must be respectful, constructive, and relevant to the topic of the guide. Do not post external links or advertisements. Before posting, consider if your comment would be better addressed by contacting our Support team or asking on our Community Site.
The Disqus commenting system for Linode Docs requires the acceptance of Functional Cookies, which allow us to analyze site usage so we can measure and improve performance. To view and create comments for this article, please update your Cookie Preferences on this website and refresh this web page. Please note: You must have JavaScript enabled in your browser.