Challenge 1: Extract and Load FHIR EHR Data

< Previous Challenge - Home - Next Challenge>

Introduction

In this challenge, you will implement the FHIR Bulk Loader function app-based event-driven architecture to ingest and load patient data in FHIR. You will generate synthetic FHIR patient data for bulk load into FHIR Server. To generate synthetic patient data, you will use the SyntheaTM Patient Generator open source Java tool to simulate patient records in FHIR format.

FHIR bulk load scenario

In this scenario, you will deploy a storage account with a BLOB container and copy Synthea generated FHIR patient data files (JSON Bundles) into it. These FHIR Bundles will be automatically ingested into a FHIR server. This bulk ingestion process will be kicked-off by an Event Grid Trigger (blobCreatedEvent) Function App as depicted below:

Description

First you will deploy Azure Health Data Services workspace and deploy a FHIR service within the workspace.

• You need to register your public client application to connect FHIR Loader and Postman desktop apps to FHIR service in Azure Health Data Services.
• Then Configure RBAC roles to assign access to the Azure Health Data Services data plane.
• After creating the new client application, you will create a new client secret.

Note: Make sure to copy and store the Client ID and Password needed in the FHIR Loader script deployment later.

You will then implement the FHIR Bulk Loader Function App solution to ingest and load Synthea generated FHIR patient data into the FHIR service in near real-time.

• Install and configure FHIR Bulk Loader with the deploy script.
• Validate your deployment, check Azure components installed:
  • Function App with App Insights and Storage
  • Function App Service plan
  • EventGrid
  • Storage Account (with containers)
  • Key Vault

To test the FHIR Bulk Loader, you will copy Synthea generated test FHIR patient data files to a specified Data Lake storage for bulk load into the FHIR service.

• Generate simulated patient data in FHIR format using SyntheaTM Patient Generator.
  • Configure the Synthea default properties for FHIR output. Below are the recommended properties setting for this challenge:
    • Set Synthea export directory: exporter.baseDirectory = ./output/fhir
    • Enable FHIR bundle export: exporter.fhir.export = true
    • Generate 1000 patient records: generate.default_population = 1000
• Load Synthea generated FHIR bundle JSON files
  • Copy from Synthea project subfolder ./output/fhir to bundles BLOB container.
  • You can copy data to Azure Storage using Azure AzCopy commandline tool or Azure Storage Explorer user interface.
• Test the results of FHIR bulk load using Postman FHIR API collection to retreive FHIR patient data loaded.
  • You need to first register your public client application to connect Postman desktop app to FHIR service in Azure Health Data Services (if not completed previously).
  • Then Configure RBAC roles to assign access to the Azure Health Data Services data plane (if not completed previously).
  • To access FHIR service using Postman, you need to import the FHIR API Postman collection and environment variables:
    • You can find the Postman template files (WTHFHIR.postman_collection.json and WTHFHIR.postman_environment.json) in the /Postman folder of the Resources.zip file provided by your coach.
    • Import the environment and collection template files into your Postman
    • Configure Postman environment variables specific to your FHIR service instance

Success Criteria

• You have provisioned FHIR service and FHIR Bulk Load environment in Azure.
• You have generated synthetic patient data in FHIR format.
• You have loaded FHIR patient data into FHIR Server.
• You have retrieved the new FHIR patient data using Postman.

Learning Resources

• What is Azure Health Data Services?
• Get started with FHIR service
• FHIR Bulk Loader
• Synthea Patient Generator
• Synthea wiki
• Copy data to Azure Storage using Azure AzCopy tool
• Copy data to Azure Storage using Azure Storage Explorer
• Register a client application in Azure Active Directory
• Import Postman data, including collections, environments, data dumps, and globals.
• Access FHIR service using Postman
• Configure RBAC roles