TIP

💡 Learn more : Use Azure Key Vault with FlexVolume (opens new window)

This post was brought to you by @kumarallamraju (opens new window)

Use Azure Key Vault to store sensitive data and access from Kubernetes Pods

As you develop and run applications in Azure Kubernetes Service (AKS), the security of Kubernetes pods is a key consideration. Your applications should be designed for the principle of least number of privileges required. Keeping private data secure is top of mind for your application. It's not a recommended practice to store sensitive data to your code or embed them in your container images. This approach would create a risk for exposure and limit the ability to rotate those credentials as the container images will need to be rebuilt.

Azure Key Vault FlexVolume for Kubernetes allows you to consume sensitive data from Azure Key Vault (like secrets, keys or certificates) and attach that data directly to Pods. You can find the OSS project here (opens new window). Azure Key Vault FlexVolume allows you to seamlessly integrate your key management systems with Kubernetes. Sensitive data like Secrets, keys, and certificates in a key management system become a volume accessible to pods. Once the volume is mounted, its data is available directly in the container filesystem for your application. Let's see it in action.

Prerequisites

If you want to follow along, you'll need the following:

An Azure subscription (If you don't have an Azure subscription, create a free account (opens new window)

Pls make sure you have an AKS cluster provisioned and running before installing the Azure Key Vault FlexVolume. Instructions to create a new AKS cluster is documented here (opens new window)

1. Deploy Key Vault FlexVolume to your AKS cluster with this command:

kubectl create -f https://raw.githubusercontent.com/Azure/kubernetes-keyvault-flexvol/master/deployment/kv-flexvol-installer.yaml

2. To validate Key Vault FlexVolume is running as expected, run the following command:

kubectl get pods -n kv

3. Using Key Vault FlexVolume

Key Vault FlexVolume offers four modes for accessing a Key Vault instance: Service Principal, Pod Identity, [VMSS User Assigned Managed Identity], [VMSS System Assigned Managed Identity].

In this blog we are going to focus on Option #1 - Service Principal

Create a service principal and read in the application ID

SP=$(az ad sp create-for-rbac --output json)
SP_ID=$(echo $SP | jq -r .appId)
SP_PASSWORD=$(echo $SP | jq -r .password)

Add your service principal credentials as Kubernetes secrets accessible by the Key Vault FlexVolume driver.

kubectl create secret generic kvcreds --from-literal clientid=<SP_ID> --from-literal clientsecret=<SP_PASSWORD> --type=azure/kv

4. Ensure this service principal has all the required permissions to access content in your Key Vault instance. If not, run the following Azure CLI commands:

# Assign Reader Role to the service principal for your keyvault
az role assignment create --role Reader --assignee <principalid> --scope /subscriptions/<subscriptionid>/resourcegroups/<resourcegroup>/providers/Microsoft.KeyVault/vaults/<keyvaultname>


# Assign key vault permissions to your service principal
az keyvault set-policy -n $KV_NAME --key-permissions get --spn <YOUR SPN CLIENT ID>
az keyvault set-policy -n $KV_NAME --secret-permissions get --spn <YOUR SPN CLIENT ID>
az keyvault set-policy -n $KV_NAME --certificate-permissions get --spn <YOUR SPN CLIENT ID>

5. I have showed an example below that mounts the volume and references the secrets stored on Azure Key Vault

apiVersion: v1
kind: Pod
metadata:
  name: nginx-flex-kv
spec:
  containers:
  - name: nginx-flex-kv
    image: nginx
    volumeMounts:
    - name: test
      mountPath: /kvmnt
      readOnly: true
  volumes:
  - name: test
    flexVolume:
      driver: "azure/kv"
      secretRef:
        name: kvcreds
      options:
        usepodidentity: "false"
        keyvaultname: "testkeyvault"
        keyvaultobjectnames: "SQL-USR;SQL-PWD;SQL-FQDN;SQL-DBNAME"
        keyvaultobjecttypes: "secret;secret;secret;secret"
        keyvaultobjectaliases: "SQL_USER;SQL_PASSWORD;SQL_SERVER;SQL_DBNAME"
        keyvaultobjectversions: "testversion"
        resourcegroup: "testresourcegroup"
        subscriptionid: "xxxx9280-xx-4xx183-xxxx-xxxxxx"
        tenantid: "xxxxdfaa-1983-4571-1111-123455d2537"

Please make sure the Azure Key Vault was created ahead of time before deploying this application. In the above example I have a keyvault named testkeyvault, 4 secrets named SQL-USR;SQL-PWD;SQL-FQDN;SQL-DBNAME and their values exists.

6. Deploy your app

kubectl create -f deployment/nginx-flex-kv.yaml

7. Validate the pod has access to the secret from key vault:

kubectl exec -it nginx-flex-kv ls /kvmnt
SQL_USER;SQL_PASSWORD;SQL_SERVER;SQL_DBNAME

This output confirm the Kubernetes Pod was able to access the sensitive data from Azure Key Vault.

Conclusion

With Azure Key Vault, you can store and regularly rotate secrets such as credentials, storage account keys, or certificates. You can integrate Azure Key Vault with an AKS cluster using a FlexVolume. The FlexVolume driver lets the AKS cluster natively retrieve credentials from Key Vault and securely provide them only to the requesting pod.

References

• Key Vault FlexVolume (opens new window)

• Azure Key Vault (opens new window)