Data will be stored on Debian VM with NFS. Provisioning persistent volumes will be automated like in the cloud. Every PVC will have his own sub directory on NFS. I will use Kubernetes NFS Subdir External Provisioner and Helm. Helm is installed on jumpbox with access to the existing cluster. Setup is for home lab, special security for NFS server will not be implemented.
Architecture:
PVC → StorageClass → NFS Subdir Provisioner → NFS VM On the VM:
apt update
apt install nfs-kernel-server -y
mkdir -p /srv/k8s
chmod 777 /srv/k8s
chown nobody:nogroup /srv/k8sIn /etc/exports add (where 172.16.18.0/24 is the subnet of the VM and k8s nodes) :
/srv/k8s 172.16.18.0/24(rw,sync,no_subtree_check,no_root_squash)And
sudo exportfs -ra
sudo systemctl restart nfs-kernel-serverOn every k8s node
sudo apt install nfs-common -yFrom jumpbox add NFS provisioner repo
helm repo add nfs-subdir https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
helm repo updateInstall with:
NFS VM : NFS server IP
Path: /srv/k8s
Nodes CIDR: 172.16.18.0/24
helm install nfs-provisioner nfs-subdir/nfs-subdir-external-provisioner \
--set nfs.server=172.16.18.19 \
--set nfs.path=/srv/k8s \
--set storageClass.name=nfs-dynamic \
--set storageClass.defaultClass=trueCheck if it works:
kubectl get podsExprecting : nfs-provisioner-nfs-subdir-external-provisioner-xxxxx Running
Check StorageClass:
kubectl get storageclassCreate test PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-dynamic-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5GiNB: Hey!!! It is you, from the future! NFS uses random ports for some internal checking. If you use firewall allow all ports to 1 IP only, or expect problems.
