TIP
🔥 Make sure you star the repo (opens new window) to keep up to date with new tips and tricks.
💡 Learn more : What is IP address 168.63.129.16? (opens new window)
This post was brought to you by Kumar Allamraju (opens new window).
# What is IP address 168.63.129.16?
Recently I was engaged to help on customer's issue in which their Azure Network Load Balancer (opens new window) is unable to reach their backend instances because the load balancer declared them as unhealthy. However they can directly reach the backend instances and able to execute the application but unable to reach the same via Azure NLB.
# Analysis
The first thing I did was to check their load balancer's health probes. However they are using "Basic Load Balancer" and the health probes are only available in the standard tier. I end up working with my Cloud Ops/Support team to see what's going on for this customer's load balancer. Further investigation revealed that customer's Network Security Group (NSG), with a rule named ‘defaultdenyinbound’ is not accepting any traffic other than their VNet IP's. Due to this rule, NLB was unable to send probes to backend instances. I recommended the customer to make the following changes
Open the Azure portal and browse to the Network Security Group (opens new window) blade assigned to your Network Load Balancer
Add an Allow rule for IP address 168.63.129.16 that has a lower number (higher priority) than the Block rule
Save changes and check the Health Probe Status by going to Load Balancer Monitoring blade to view the backend pool health
The public IP address 168.63.129.16 is used in all Azure public regions and all national clouds. This special public IP address is owned by Microsoft and will not change. It is allowed by the default network security group rule. We recommend that you allow this IP address in any local firewall policies in both inbound and outbound directions. The communication between this special IP address and the resources is safe because only the internal Azure platform can source a message from this IP address. If this address is blocked, unexpected behavior can occur in a variety of scenarios.
Azure Load Balancer probes originates from this IP address. If customer's block this IP address, their probes will fail leading to the above situation. The customer is convinced with the above explanation. After allowing this IP in their local firewall policies, they are able to reach the backend instances via Azure NLB and the problem is solved.