Open data Leverage open data to normalize and enhance your FinOps reporting.
On this page
Reporting is the life-blood of any FinOps initiative. And your reports are only as good as your data. This is why data ingestion and normalization is such an important part of FinOps (and any big data effort). The following datasets can be used to clean and normalize your data as part of data ingestion, reporting, or other solutions.
📏 Pricing units
Microsoft Cost Management uses the UnitOfMeasure
column to indicate how each charge is measured. This can be in singular or distinct units or can be grouped into chunks based on applicable block pricing rules. As a string, the UnitOfMeasure
column can be challenging to parse and handle all the different permutations and inconsistencies. The Pricing units file provides a list of values you may find within common cost-related datasets (e.g., Cost Management exports and price sheets) along with their related distinct unit and block size or scaling factor to compare pricing to usage units.
Sample data:
UnitOfMeasure | AccountTypes | PricingBlockSize | DistinctUnits |
---|---|---|---|
1 Hour | MCA, EA | 1 | Hours |
10000 GB | EA | 10000 | GB |
150 Hours | EA | 150 | Hours |
200 /Hour | EA | 200 | Units/Hour |
5 GB | MCA, EA | 5 | GB |
5000000 Requests | EA | 5000000 | Requests |
744 Connections | EA | 744 | Connections |
A few important notes about the data:
- Meter names are not included to keep the file size down.
- The default unit type is “Units”.
- Some default units may include a more specific unit in the meter name, which is not accounted here since meter names aren’t included.
- Marketplace meters are not included due to inconsistencies that would impact data size.
In the Cost Management FOCUS dataset,
UnitOfMeasure
is renamed tox_PricingUnitDescription
. BothPricingUnit
andUsageUnit
in FOCUS are set to theDistictUnits
column.
🗺️ Regions
Microsoft Cost Management provides various values for resource locations that are occasionally inconsistent due to different underlying systems providing the data. The Regions file provides a list of values you may find within common cost-related datasets (e.g., Cost Management exports and price sheets) along with their related Azure region IDs and names.
Sample data:
OriginalValue | RegionId | RegionName |
---|---|---|
ap east | eastasia | East Asia |
ca central | canadacentral | Canada Central |
de north | germanynorth | Germany North |
no west | norwaywest | Norway West |
tw north | taiwannorth | Taiwan North |
Convert region values to lowercase before mapping. This helps reduce duplication and speed up the mapping process.
📚 Resource types
Azure resource types are a semi-readable code that represents what kind of resource it is. Currently, there’s no mapping of the resource type to a user-friendly string, description, or its icon. The ResourceTypes file provides a list of resource type values you’ll find in the Azure portal along with their display names, description, and a link to the icon, when available.
Sample data:
ResourceType | Singular Display Name | Plural Display Name | Lower Singular Display Name | Lower Plural Display Name |
---|---|---|---|---|
microsoft.compute/virtualmachines | Virtual machine | Virtual machines | virtual machine | virtual machines |
microsoft.insights/workbooks | Azure Workbook | Azure Workbooks | azure workbook | azure workbooks |
microsoft.logic/workflows | Logic app | Logic apps | logic app | logic apps |
microsoft.network/virtualnetworks | Virtual network | Virtual networks | virtual network | virtual networks |
microsoft.recoveryservices/vaults | Recovery Services vault | Recovery Services vaults | recovery services vault | recovery services |
microsoft.search/searchservices | Search service | Search services | search service | search services |
microsoft.sql/servers | SQL server | SQL servers | SQL server | SQL servers |
microsoft.sql/servers/databases | SQL database | SQL databases | SQL database | SQL databases |
microsoft.web/sites | App Service web app | App Service web apps | app service | app services |
Convert resource type values to lowercase before mapping. This helps reduce duplication and speed up the mapping process.
Download CSV Download JSON See PowerShell
🎛️ Services
In Microsoft Cost Management, ConsumedService
represents the primary service or resource provider of the resource you used. This is roughly the same as ServiceName
in FOCUS. In some cases, multiple services share the same resource provider, so we’re using the ConsumedService
and ResourceType
columns to map to ServiceName
and ServiceCategory
values for use within FOCUS.
Sample data:
ConsumedService | ResourceType | ServiceName | ServiceCategory | ServiceSubcategory | PublisherName | PublisherType |
---|---|---|---|---|---|---|
microsoft.compute | microsoft.compute/virtualmachines | Virtual Machines | Compute | Virtual Machines | Microsoft | Cloud Provider |
microsoft.documentdb | microsoft.documentdb/databaseaccounts | Cosmos DB | Databases | NoSQL Databases | Microsoft | Cloud Provider |
microsoft.kusto | microsoft.kusto/clusters | Azure Data Explorer | Analytics | Analytics Platforms | Microsoft | Cloud Provider |
microsoft.network | microsoft.network/virtualnetworks | Virtual Network | Networking | Networking Infrastructure | Microsoft | Cloud Provider |
microsoft.storage | microsoft.storage/storageaccounts | Storage Accounts | Storage | Storage Platforms | Microsoft | Cloud Provider |
A few important notes about the data:
ConsumedService
andResourceType
values are all lowercased to avoid case sensitivity issues.ServiceName
values should match the product marketing name for the closest possible service. Some services reuse resource types and cannot be distinguished from the resource type alone (e.g., Azure functions will show as App Service).ServiceCategory
values are aligned with the allowed values in FOCUS.
Most mappings can rely on resource type alone. In a future update, we will merge this list with Resource types to provide only a single dataset. Currently, the only known case where resource type is shared that ConsumedService can help identify is for Microsoft Defender for Cloud. To simplify your mapping, you can only map those 5 rows and rely on a resource type mapping for everything else.
⬇️ Dataset examples
The following files are examples of what you will find when you export data from Microsoft Cost Management. These files are provided to help you understand the data structure and format. They are from an Enterprise Agreement (EA) demo account and are not intended to be used for ingestion or reporting.
- Cost and usage
- Actual (billed) (
2021-10-01
) - Amortized (
2021-10-01
) - FOCUS (
1.0-preview(v1)
)
- Actual (billed) (
- Prices (
2023-05-01
) - Reservation details (
2023-03-01
) - Reservation transactions (
2023-05-01
) - Reservation recommendations (
2023-05-01
)
📃 Dataset metadata
Given each dataset uses different columns and data types, FOCUS has defined metadata schema to describe the dataset. Dataset metadata includes general information about the data like the data generator, schema version, and columns included in the dataset.
Sample data:
ColumnName | DataType | Description |
---|---|---|
BilledCost | Decimal | A charge serving as the basis for invoicing, inclusive of all reduced rates and discounts while excluding the amortization of upfront charges (one-time or recurring). |
BillingAccountId | String | Unique identifier assigned to a billing account by the provider. |
BillingAccountName | String | Display name assigned to a billing account. |
BillingCurrency | String | Currency that a charge was billed in. |
BillingPeriodEnd | DateTime | End date and time of the billing period. |
BillingPeriodStart | DateTime | Beginning date and time of the billing period. |
Metadata is available for the following datasets:
- Cost and usage
- FOCUS 1.0 – Learn more
- FOCUS 1.0-preview(v1) – Learn more
🙋♀️ Looking for more?
We’d love to hear about any datasets you’re looking for. Create a new issue with the details that you’d like to see either included in existing or new datasets.
🧰 Related tools
- 📊 Power BI reports – Accelerate your reporting with Power BI starter kits.
- 🖥️ PowerShell module – Automate and manage FinOps solutions and capabilities.