How to Configure MicroK8s Nodes in a Virtual Network

This guide addresses an issue I encountered while managing a highly available MicroK8s cluster with at least three nodes across multiple locations. Although the nodes have IP addresses from their physical network, they also receive IPs from their virtual network, which is essential for inter-node communication. The problem: after some time, the nodes seem to lose the ability to locate each other or even resources on themselves. In my case, the DHCP server of the physical network reassigned them new IP addresses on their physical connection.

"If you are using a virtual network for your MicroK8s cluster, ensure that this network and its node IPs are set as primary in your configuration. This ensures that internal networking does not fallback to searching for the right config on the physical address or relies on cluster certificates bound to the physical server IP address, but to the actual VPN IP address the nodes are communicating with.

What is my IP?

To make sure that MicroK8s is using the right IPs you have to check your network adapters.

# wt0: in my case, this is the name of a Wireguard based network adapter
# Other vpn adapters might have different names, if you are using OpenVPN, IPSec, etc.
ip addr show dev wt0 | grep inet

# Result
inet 100.100.10.140/16 brd 100.84.255.255 scope global wt0

So in my case the IP address within the virtual network is 100.100.10.140 and I need to make sure with my VPN application that it does not change, if that is not the default.

Configuring microk8s node IPs

Now that we have the actual IP that we want MicroK8s to work with, we have to set it in two places. First stop your running microk8s instance with microk8s stop.

The first file is
/var/snap/microk8s/current/args/kubelet

# Add this to the end of the file or check for an existing entry to replace with
# In my case it is --node-ip=100.100.10.140
--node-ip=THE.IP.FROM.ABOVE

The second file is
/var/snap/microk8s/current/args/kube-apiserver

# Add this to the end of the file or check for an existing entry to replace with
# In my case it is --advertise-address=100.100.10.140
--advertise-address=THE.IP.FROM.ABOVE

Now make sure to microk8s start your node again. To verify if your changes have been applied correctly, run the following command: microk8s kubectl get nodes -o wide

You should see your IP in the column INTERNAL IP listed for your verification. Remember, this process must be done on every node in the cluster within the virtual network. For further details the Microk8s documentation provides thorough information on configuring the IP:

MicroK8s - Configure host interfaces used by MicroK8s

Thanks for reading this short instruction! If you found this helpful, I'd be happy for a reaction or a comment. If you have some questions, feel free to ask :)