IP Address Planning

Proper IP address planning is crucial for the successful deployment and operation of your Azure Kubernetes Service (AKS) cluster. Without careful planning, you may encounter issues such as IP exhaustion, routing conflicts, and difficulties in scaling your cluster. This is particularly important if you are using a flat networking model like Azure CNI Pod Subnet, where pod IPs are assigned directly from a subnet within your Azure VNet. If using an overlay model like Azure CNI Overlay or Azure CNI Powered by Cilium, while the IP planning is less critical, it still requires attention to ensure optimal performance and scalability.

Key Considerations for IP Address Planning

  1. Estimate Maximum Cluster Size
    • Calculate the maximum number of nodes and pods you expect.
    • Example: 10 nodes × 30 pods per node = 300 pod IPs needed (plus node IPs).
  2. Subnet Sizing
    • Use a subnet large enough to accommodate all node and pod IPs.
    • Azure recommends a /16 or /17 subnet for production clusters.
    • Avoid using small subnets (e.g., /24) unless for testing.
  3. Separate Subnets
    • Use dedicated subnets for AKS node pools.
    • Isolate system and user node pools for security and manageability.
  4. Avoid Overlapping IP Ranges
    • Ensure your AKS subnet does not overlap with other VNets, on-premises networks, or VPNs.
  5. Plan for Future Growth
    • Leave room for scaling out node pools and adding new services.

Cluster and Node Upgrades

When updating Kubernetes versions or node images, AKS may temporarily scale out your node pool to maintain availability. This results in additional nodes being created, each requiring its own set of pre-allocated pod IPs. If your subnet has limited available IP addresses, this temporary increase can lead to IP exhaustion, causing upgrade failures or scheduling issues. To avoid this, always ensure your subnet includes a buffer of extra IPs to accommodate upgrade-related scaling.

IP Addressing in Flat vs Overlay Networking

Flat Networking (Azure CNI Pod Subnet)

  • Requires careful planning of the subnet size to accommodate both node and pod IPs.
  • Each pod gets an IP from the VNet, so the subnet must be large enough to handle the expected number of pods.
  • Example: If you expect 100 nodes with 30 pods each, you need at least a /20 subnet (4094 IPs) for your pod subnet, to avoid exhaustion and accommodate scale-out for upgrades and maintenance.

Overlay Networking (Azure CNI Overlay or Cilium)

  • Uses a separate overlay CIDR for pods, which can be much larger than the VNet.
  • Reduces the need for large subnets in the VNet, as pods are not directly routable from the VNet.
  • Example: You can use a /16 overlay CIDR for pods, allowing for up to 65,536 pod IPs without impacting the VNet size.