Tip 253 - Use Ingress Controller to access resources in an Azure Virtual Network

TIP

💡 Learn more : Ingress Controller in Azure Kubernetes Service (opens new window)

This article was brought to you by @kumarallamraju (opens new window)

# Use Ingress Controller to access resources in an Azure Virtual Network

# Intro

In a recent customer engagement, they wanted to access all the AKS services via an Azure virtual network. This customer's application is not an external facing and needs to be on a corporate network to access their resources (e.g. HR, payroll,procurement etc..) If you are familiar with the Kubernetes eco system, we can use NGINX ingress controller to expose Kubernetes services on an external IP address. Azure Kubernetes Services comes to the rescue. AKS provides an option to deploy your NGINX ingress controller on an internal network which keeps the resources accessible only on an internal network and can be accessible via Express Route or VPN.

In few simple steps, let's understand the process to make this happen.

# Prerequisites

Please make sure you have an AKS cluster provisioned and running before proceeding further. Instructions to create a new AKS cluster is documented here (opens new window)

This article uses Helm to install the NGINX ingress controller, and a sample web app. You need to have Helm initialized within your AKS cluster and using a service account for Tiller. For more information on configuring and using Helm, see Install applications with Helm in AKS (opens new window)

# 1. Create an NGINX Ingress Controller

Create a file named internal-ingress.yaml using the following example manifest file. This example assigns 10.240.0.42 to the loadBalancerIP resource. Provide your own internal IP address for use with the ingress controller. Make sure that this IP address is not already in use within your virtual network.

controller:
  service:
    loadBalancerIP: 10.240.0.42
    annotations:
      service.beta.kubernetes.io/azure-load-balancer-internal: "true"

1
2
3
4
5

# 2. Create a separate namespace and ingress controller by passing the above file name.

# Create a namespace for your ingress resources
kubectl create namespace ingress-basic

# Use Helm to deploy an NGINX ingress controller
helm install stable/nginx-ingress \
    --namespace ingress-basic --name mynginx-ingress \
    --set controller.replicaCount=1 \
    --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
    --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux \
    -f internal-ingress.yaml #[this is important]

1
2
3
4
5
6
7
8
9
10

# 3. Check the status of nginx controller

kubectl get service -l app=nginx-ingress --namespace ingress-basic

It takes a minute or two to see assigned private IP in the "EXTERNAL-IP" section.

1
2
3

# 4. Let's create some ingress rules to see the example in action.

In this example, Helm is used to deploy two instances of a simple 'Hello world' application.

helm repo add azure-samples https://azure-samples.github.io/helm-charts/

1
2

# 5. Create the first demo application from a Helm chart with the following command:

helm install azure-samples/aks-helloworld --namespace ingress-basic

# 6. Now install a second instance of the demo application.

For the second instance, you specify a new title so that the two applications are visually distinct. You also specify a unique service name:

helm install azure-samples/aks-helloworld \
    --namespace ingress-basic \
    --set title="Allamraju's 2nd AKS Ingress Demo" \
    --set serviceName="ingress-demo"

1
2
3
4

# 7. Create an Ingress Route

Both applications are now running on your Kubernetes cluster. To route traffic to each application, create a Kubernetes ingress resource. The ingress resource configures the rules that route traffic to one of the two applications.

In the following example, traffic to the address http://10.240.0.42/ is routed to the service named aks-helloworld. Traffic to the address http://10.240.0.42/hello-world-two is routed to the ingress-demo service.

Create a file named hello-world-ingress-internalIP.yaml and copy in the following example YAML.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-world-ingress
  namespace: ingress-basic
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: aks-helloworld
          servicePort: 80
        path: /(.*)
      - backend:
          serviceName: ingress-demo
          servicePort: 80
        path: /hello-world-two(/|$)(.*)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# 8. Deploy this resource to your Kubernetes cluster using kubectl command

kubectl apply -f hello-world-ingress.yaml

# 9. Time to validate the ingress controller

To test the routes for the ingress controller, browse to the two applications pointing your web browser to http://10.240.0.42. But to resolve this IP you need to access this IP via Express Route or VPN from your on-premises network. If needed, you can quickly test this internal-only functionality from a pod on the AKS cluster. Create a test pod and attach a terminal session to it:

kubectl run -it --rm aks-ingress-test --image=debian --namespace ingress-basic

apt-get update && apt-get install -y curl

1
2
3

curl -L http://10.240.0.42

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <link rel="stylesheet" type="text/css" href="/static/default.css">
    <title>Welcome to Azure Kubernetes Service (AKS)</title>
[...]

1
2
3
4
5
6

curl -L -k http://10.240.0.42/hello-world-two

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <link rel="stylesheet" type="text/css" href="/static/default.css">
    <title>KS Ingress Demo</title>
[...]

1
2
3
4
5
6

Once you are done validating this functionality, please do not forget to clean up the resources

helm repo remove azure-samples

helm ls --all mynginx-ingress --namespace ingress-basic
helm del --purge mynginx-ingress

kubectl delete -f hello-world-ingress-internalIP.yaml

kubectl delete namespace ingress-basic

1
2
3
4
5
6
7
8

# Conclusion

By default, an NGINX ingress controller is created with a dynamic public IP address assignment. A common configuration requirement is to use an internal, private network and IP address. The approach discussed above allows you to restrict access to your services to internal users, with no external access via public internet.

# References

← Tip 245 - Use Azure Key Vault to store sensitive data and access from Kubernetes Pods
Tip 253 - Use Ingress Controller to access resources in an Azure Virtual Network →