Kubernetes cluster for a small lab installed on Debian virtual machines. Set-up just one instance first, clone it later. One control plane, two worker nodes. Official Documentation
Requirements:
- Debian 13 trixie
- Network connectivity between the VMs. Every VM has unique MAC address, IP address, hostname
- 2 GB or more of RAM per machine (any less will leave little room for your apps).
- 2 CPUs or more for control plane machines.
Prepare Debian
Disable swap. sudo mount -av or reboot for changes to take effect
sudo sed -i '/swap/s/^/#/' /etc/fstab
Set static IP or use route to set it static
sudo vim /etc/network/interfacesset hostname. Do not forget to set the same name in /etc/hosts. Master for control plane, worker1 and worker2 for worker nodes
sudo vim /etc/hostnameSome kernel black magic
overlay: Container filesystem support.br_netfilter: Bridge traffic passes through iptables.sysctl settings: Enable packet filtering and routing for pod traffic.sysctl --system: Applies the changes so networking actually works.
1. sudo modprobe overlay && sudo modprobe br_netfilter && \
sudo tee /etc/sysctl.d/kubernetes.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF
sudo sysctl --systemSet up container run times
AWS uses containerd so am I too. check this and this. Option 1 with official binaries
Download the archive
wget https://github.com/containerd/containerd/releases/download/v2.2.1/containerd-2.2.1-linux-amd64.tar.gzExtract
sudo tar Cxzvf /usr/local containerd-2.2.1-linux-amd64.tar.gzConfigure systemd service. Paste line by line
wget https://raw.githubusercontent.com/containerd/containerd/main/containerd.service
cp containerd.service /etc/systemd/system/containerd.service
systemctl daemon-reload
systemctl enable --now containerdset systemd cgroup in containerd. Find this section in config.toml (search SystemdCgroup) set it to : SystemdCgroup = true
sudo mkdir -p /etc/containerd
sudo containerd config default > config.toml
sudo mv config.toml /etc/containerd/Restart the service
sudo systemctl restart containerd.serviceRunc. Paste line by line
wget https://github.com/opencontainers/runc/releases/download/v1.4.0/runc.amd64
sudo install -m 755 runc.amd64 /usr/local/sbin/runc
wget https://github.com/containernetworking/plugins/releases/download/v1.9.0/cni-plugins-linux-amd64-v1.9.0.tgz
sudo mkdir -p /opt/cni/bin
sudo tar Cxzvf /opt/cni/bin/ cni-plugins-linux-amd64-v1.9.0.tgzInstall kubeadm
Documentation here. Paste line by line
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.35/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb \[signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg\] https://pkgs.k8s.io/core:/stable:/v1.35/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable --now kubeletKubeadm is now installed. Instance is ready to be used for control plane and node (workers). Clone it, according to your needs.
Start cluster creation
–control-plane-endpoint Can be control plane IP or Load Balancer like HAProxy
–pod-network-cidr CIDR that will be used inside the cluster.
sudo kubeadm init --pod-network-cidr 10.244.0.0/16 --control-plane-endpoint "192.168.77.78:6443" --upload-certs --v=5Expected result
to start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes running the following command on each as root:
kubeadm join 192.168.77.78:6443 --token rl33j8.fs0wxtu4ry1f2dko \
--discovery-token-ca-cert-hash sha256:f3808447540c84c112b1d33b5e4b161091f38c0b215a84a72c66b9c4b50b6301 \
--control-plane --certificate-key 6ef0ba2a9dec00638c5a3d08e660bbbf807e40a6d529acbb82eaf9bc4bc09f7e
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.77.78:6443 --token rl33j8.fs0wxtu4ry1f2dko \
--discovery-token-ca-cert-hash sha256:f3808447540c84c112b1d33b5e4b161091f38c0b215a84a72c66b9c4b50b6301 Join worker node to the cluster (cloned VM, remember?!)
kubeadm join 192.168.77.78:6443 --token rl33j8.fs0wxtu4ry1f2dko \
--discovery-token-ca-cert-hash sha256:f3808447540c84c112b1d33b5e4b161091f38c0b215a84a72c66b9c4b50b6301
Back on control plane node. Configure kubectl
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/configAdd some networking. Pods need something to talk each other. Calico manifest is easy. Remove comments of
– name: CALICO_IPV4POOL_CIDR
value: “192.168.0.0/16
and change 192.168.0.0/16 with 10.244.0.0/16
wget https://docs.projectcalico.org/manifests/calico.yamlApply the manifest
kubectl apply -f calico.yaml
Run first pod
kubectl run nginx \
--image=nginx \
--restart=Never
It’s is working:)
If you need to add new nodes in future (more than 24 hours) you need to generate new tokens on master control plane
kubeadm token create --print-join-command