
Hi guys,
One of the main networking challenges in Kubernetes clusters is IP address consumption. With the traditional Azure CNI networking model, every Pod is assigned an IP address from the Azure Virtual Network (VNet). As the cluster grows, this can quickly exhaust the available IP address space, especially in large enterprise environments.
Azure CNI Overlay addresses this challenge by separating the network used by the nodes from the network used by the Pods:
- Nodes use IP addresses from the Azure Virtual Network.
- Pods use an independent Overlay network (for example,
192.168.0.0/16). - Azure automatically manages routing between nodes, allowing Pods to communicate transparently regardless of where they are running.
Architecture Flow
- The user accesses the application through Azure Application Gateway.
- The request is routed to the Azure Kubernetes Service (AKS) cluster.
- The Frontend Pod communicates with the Backend Pod using the Overlay network.
- The Backend accesses Azure private services such as Azure SQL Database, Storage Account, and Azure Key Vault through Private Endpoints.
- When Internet access is required, outbound traffic is routed through Azure Firewall and Azure NAT Gateway, where security policies and SNAT are applied.
- The response is then returned to the user following the same path.
Key Components
- Azure Application Gateway (Ingress)
- Azure Kubernetes Service (AKS)
- Azure CNI Overlay
- Azure Firewall
- Azure NAT Gateway
- Azure SQL Database (Private Link)
- Azure Storage Account
- Azure Key Vault
- Azure Container Registry (ACR)
- Azure Monitor
- Prometheus
- Grafana
- Azure DevOps Pipelines
Benefits of Azure CNI Overlay
- Significantly reduces VNet IP address consumption.
- Enables clusters to scale to thousands of Pods without continuously expanding subnet sizes.
- Maintains native integration with Azure networking services.
- Fully compatible with Private Link, Network Policies (Cilium), and Azure Firewall.
- Eliminates the need to manage User Defined Routes (UDRs), unlike Kubenet.
- Simplifies the design of enterprise Azure Landing Zones.
Key Takeaway
The most important difference is that only the AKS nodes consume IP addresses from the Azure Virtual Network, while Pods use an Azure-managed Overlay network. This architecture enables highly scalable, secure, and efficient AKS platforms while optimizing IP address utilization and maintaining seamless integration with the Azure ecosystem.