FinOps hub template Behind the scenes peek at what makes up the FinOps hub template, including inputs and outputs.
On this page
This template creates a new FinOps hub instance.
FinOps hubs include:
- Data Lake storage to host cost data.
- Data Factory for data processing and orchestration.
- Key Vault for storing secrets.
To use this template, you will need to create a Cost Management export that publishes cost data to the
msexports
container in the included storage account. See Create a new hub for details.
📋 Prerequisites
Please ensure the following prerequisites are met before deploying this template:
-
You must have the following permissions to create the deployed resources.
Resource Minimum RBAC Deploy and configure Data Factory Data Factory Contributor Deploy Key Vault Key Vault Contributor Configure Key Vault secrets Key Vault Administrator Create managed identity Managed Identity Contributor Deploy and configure storage Storage Account Contributor Create a subscription or resource group cost export1 Cost Management Contributor Create an EA billing cost export1 Enterprise Reader, Department Reader, or Enrollment Account Owner (Learn more) Create an MCA billing cost export1 Contributor Read blob data in storage2 Storage Blob Data Contributor 1. Cost Management permissions must be assigned on the scope where you want to export your costs from. . 2. Blob data permissions are required to access exported cost data from Power BI or other client tools.
-
The Microsoft.EventGrid resource provider must be registered in your subscription. See Register a resource provider for details.
If you forget this step, the deployment will succeed, but the pipeline trigger will not be started and data will not be ready. See Troubleshooting Power BI reports for details.
📥 Parameters
Parameter | Type | Description | Default value |
---|---|---|---|
hubName | String | Optional. Name of the hub. Used to ensure unique resource names. | "finops-hub" |
location | String | Optional. Azure location where all resources should be created. See https://aka.ms/azureregions. | (resource group location) |
storageSku | String | Optional. Storage SKU to use. LRS = Lowest cost, ZRS = High availability. Note Standard SKUs are not available for Data Lake gen2 storage. Allowed: Premium_LRS , Premium_ZRS . | Premium_LRS |
tags | Object | Optional. Tags to apply to all resources. We will also add the cm-resource-parent tag for improved cost roll-ups in Cost Management. | |
exportScopes | Array | Optional. List of scope IDs to create exports for. |
🎛️ Resources
The following resources are created in the target resource group during deployment.
Resources use the following naming convention: <hubName>-<purpose>-<unique-suffix>
. Names are adjusted to account for length and character restrictions. The <unique-suffix>
is used to ensure resource names are globally unique where required.
<hubName>store<unique-suffix>
storage account (Data Lake Storage Gen2)- Blob containers:
msexports
– Temporarily stores Cost Management exports.ingestion
– Stores ingested data.In the future, we will use this container to stage external data outside of Cost Management.
config
– Stores hub metadata and configuration settings. Files:settings.json
– Hub settings.
- Blob containers:
<hubName>-engine-<unique-suffix>
Data Factory instance- Pipelines:
msexports_ExecuteETL
– Triggers the ingestion process for Cost Management exports to account for Data Factory pipeline trigger limits.msexports_ETL_transform
– Converts Cost Management exports into parquet or gzipped CSV and removes historical data duplicated in each day’s export.
- Triggers:
msexports_FileAdded
– Triggers themsexports_ExecuteETL
pipeline when Cost Management exports complete.
- Pipelines:
<hubName>-vault-<unique-suffix>
Key Vault instance- Secrets:
- Data Factory system managed identity
- Secrets:
In addition to the above, the following resources are created to automate the deployment process. The deployment scripts should be deleted automatically but please do not delete the managed identities as this may cause errors when upgrading to the next release.
- Managed identities:
<storage>_blobManager
(Storage Blob Data Contributor) – Uploads the settings.json file.<datafactory>_triggerManager
(Data Factory Contributor) – Stops triggers before deployment and starts them after deployment.
- Deployment scripts (automatically deleted after a successful deployment):
<datafactory>_stopHubTriggers
– Stops all triggers in the hub using the triggerManager identity.<datafactory>_startHubTriggers
– Starts all triggers in the hub using the triggerManager identity.uploadSettings
– Uploads the settings.json file using the blobManager identity.
📤 Outputs
Output | Type | Description |
---|---|---|
name | String | Name of the deployed hub instance. |
location | String | Azure resource location resources were deployed to. |
dataFactoryName | String | Name of the Data Factory. |
storageAccountId | String | The resource ID of the deployed storage account. |
storageAccountName | String | Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data. |
storageUrlForPowerBI | String | URL to use when connecting custom Power BI reports to your data. |