Configuration

Configuration#

The hastegeo.core.config module provides environment-aware configuration for all HASTE services. It reads environment variables to configure storage backends, queue names, paths, and credentials.

Key Classes#

Config — Main configuration class with static methods for accessing environment settings
StorageType (Enum) — Storage backend types: LOCAL, BLOB, COSMOS, DATALAKE, POSTGRES
ArtifactTypes (Enum) — Artifact naming templates for pre/post event imagery, model artifacts, etc.
InviteConfig (NamedTuple) — Configuration for the user invitation system

Environment Variables#

The Config class reads from these key environment variables (see local.settings.example.jsonc):

Variable	Description
`DATA_PATH`	Base path for local data storage
`TEMP_DATA_PATH`	Temporary processing directory
`AZURE_STORAGE_CONNECTION_STRING`	Azure Blob/Queue Storage connection
`COSMOS_CONNECTION_STRING`	Azure CosmosDB connection
`METADATA_STORAGE_TYPE`	Backend for metadata (`blob`, `localfilesystem`, `cosmos`, etc.)
`IMAGERY_STORAGE_TYPE`	Backend for imagery files
`ARTIFACT_STORAGE_TYPE`	Backend for model artifacts
`IMAGE_QUEUE`, `TRAIN_QUEUE`, etc.	Queue names for async processing

class hastegeo.core.config.ArtifactTypes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: Enum

Enumeration of artifact types with template-based naming.

Defines standardized naming templates for various data artifacts generated and stored throughout the HASTE workflow. Each template uses string substitution for dynamic naming based on project and model identifiers.

Template Variables:

${projectId}: Unique project identifier
${imageLayerId}: Unique image layer identifier
${modelName}: Model identifier/name

Artifact Categories:

PRE_EVENT_*: Pre-disaster imagery and derivatives
POST_EVENT_*: Post-disaster imagery and derivatives
BUILDING_FOOTPRINTS: Cached Overture Maps building footprints, scoped to the image layer’s AOI. Generated during imageryprep so the inference workflow can reuse the same set across multiple model runs.
VALID_AREA_MASK: GeoJSON FeatureCollection of the valid-data polygon derived from the post-event mosaic — i.e. the imagery’s actual AOI excluding nodata. Same polygon used to bbox-filter Overture; surfaced as a downloadable artifact for users.
INFERENCE_*: Model inference outputs
MODEL_*: Model artifacts and checkpoints
VISUALIZER: Visualization-ready outputs

BUILDING_EMBEDDINGS = <string.Template object>#

BUILDING_FEATURES_SIDECAR = <string.Template object>#

BUILDING_FOOTPRINTS = <string.Template object>#

BUILDING_PMTILES = <string.Template object>#

BUILDING_PREDICTIONS_GPKG = <string.Template object>#

INFERENCE_ARTIFACTS_ZIP = <string.Template object>#

INFERENCE_GPKG = <string.Template object>#

MODEL_ARTIFACTS_ZIP = <string.Template object>#

POST_EVENT_MOSAIC = <string.Template object>#

POST_EVENT_PREVIEW = <string.Template object>#

POST_EVENT_PROCESSED_COG = <string.Template object>#

POST_EVENT_RAW = <string.Template object>#

PRE_EVENT_MOSAIC = <string.Template object>#

PRE_EVENT_PREVIEW = <string.Template object>#

PRE_EVENT_PROCESSED_COG = <string.Template object>#

PRE_EVENT_RAW = <string.Template object>#

TRAINING_ARTIFACTS_ZIP = <string.Template object>#

VALID_AREA_MASK = <string.Template object>#

VISUALIZER = <string.Template object>#

class hastegeo.core.config.Config(env=None)[source]#

Bases: object

Configuration class with environment-specific settings.

The Config class manages all configuration settings for the HASTE application, including storage backends, queue configurations, and environment-specific parameters. It loads settings from environment variables and provides typed access to configuration values.

Parameters:: env (str, optional) – Environment name (‘dev’, ‘test’, ‘prod’). Defaults to value from ENV environment variable or ‘dev’.

env#

Current environment name.

Type:: str

DEBUG#

True if running in development environment.

Type:: bool

TESTING#

True if running in test environment.

Type:: bool

DATA_DIR#

Primary data directory path.

Type:: str

TEMP_DIR#

Temporary data directory path.

Type:: str

storage_type#

Type of metadata storage backend.

Type:: str

artifact_storage_type#

Type of artifact storage backend.

Type:: str

runner_type#

Type of task runner backend (‘azure_batch’).

Type:: str

Example

>>> config = Config('prod')
>>> data_types = config.get_metadata_types()
>>> storage_config = config.storage_config

property INVITE#

__init__(env=None)[source]#

Initialize configuration with environment-specific settings.

Parameters:: env (str, optional) – Environment name. Defaults to ENV environment variable or ‘dev’.

static get_artifact_types()[source]#

Get enumeration of available artifact types.

Returns:

Enumeration containing template-based artifact type: definitions for various data artifacts (raw imagery, mosaics, model artifacts, etc.).

Return type:

ArtifactTypes

static get_azure_batch_config()[source]#

Get Azure Batch configuration for training and inference workloads.

This method provides environment-specific Azure Batch configuration including VM specifications, container registry settings, and pool configurations. The configuration varies between development and production environments, with different VM sizes and operating system images optimized for each.

Returns:

Azure Batch configuration containing:

account_name: Azure Batch account name
batch_url: Azure Batch service URL
vm_size: Virtual machine size (GPU-enabled for ML workloads)
pool_id: Batch pool identifiers for training and imagery processing
registry_server: Container registry server URL
docker_image: Docker image for containerized workloads
And other batch-specific configuration parameters

Return type:

dict

Environment Variables:

AZURE_BATCH_ACCOUNT_NAME: Batch account name
AZURE_BATCH_ACCOUNT_KEY: Batch account access key
AZURE_BATCH_VM_SIZE: Override default VM size
env: Environment type (‘dev’ uses NC6S_V3, ‘prod’ uses A100)

Note

Development environment uses Ubuntu 20.04 with Standard_NC6S_V3 VMs, while production uses Ubuntu 22.04 with Standard_NC24ads_A100_v4 VMs that have pre-installed GPU drivers.

static get_data_formats()[source]#

Get enumeration of supported data formats.

Returns:

DataFormats enumeration containing supported file formats: (JSON, TIF, TIFF).

Return type:

Enum

static get_metadata_types()[source]#

Get enumeration of available metadata types.

Returns:

DataTypes enumeration containing all supported metadata types: including PROJECT, IMAGELAYER, LABELS, USERS, CONFIG, MODEL, etc.

Return type:

Enum

Example

>>> types = Config.get_metadata_types()
>>> project_type = types.PROJECT.value  # 'project'

static get_queue_config()[source]#

Get Azure Queue Storage configuration from environment variables.

Returns:

Queue configuration including connection strings and queue names.: Contains keys for various queues (image, train, inference, stats, zip).

Return type:

dict

static get_status_types()[source]#

Get enumeration of available status types for jobs and tasks.

Returns:

StatusTypes enumeration containing standard status values

used throughout the HASTE system for tracking job and task states.

Available statuses: - PENDING: Job is queued and waiting to start - IN_PROGRESS: Job is currently executing - COMPLETED: Job finished successfully - FAILED: Job encountered an error and failed - CANCELLED: Job was cancelled by user or system

Return type:

Enum

Example

>>> status_types = Config.get_status_types()
>>> current_status = status_types.IN_PROGRESS.value  # 'InProgress'

static get_user_roles()[source]#

Get enumeration of supported user roles.

Returns:

UserRoles enumeration containing supported user roles: (ADMIN, CONTRIBUTOR, VIEWER).

Return type:

Enum

static get_user_statuses()[source]#

Get enumeration of supported user status types.

Returns:

UserStatus enumeration containing supported user status types: (ACTIVE, PENDING, DELETED).

Return type:

Enum

class hastegeo.core.config.InviteConfig(STATIC_APP_SUBSCRIPTION_ID, STATIC_APP_RESOURCE_GROUP, STATIC_APP_NAME, STATIC_APP_DOMAIN, EMAIL_CONNECTION_STRING, EMAIL_SENDER, DEFAULT_USER_ROLES)[source]#

Bases: NamedTuple

DEFAULT_USER_ROLES: list[str]#: Alias for field number 6

EMAIL_CONNECTION_STRING: str#: Alias for field number 4

EMAIL_SENDER: str#: Alias for field number 5

STATIC_APP_DOMAIN: str#: Alias for field number 3

STATIC_APP_NAME: str#: Alias for field number 2

STATIC_APP_RESOURCE_GROUP: str#: Alias for field number 1

STATIC_APP_SUBSCRIPTION_ID: str#: Alias for field number 0

class hastegeo.core.config.StorageType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: Enum

Enumeration of supported storage backend types.

Defines the available storage backends that can be used for metadata and artifact storage in the HASTE system.

Values:: LOCAL: Local filesystem storage BLOB: Azure Blob Storage COSMOS: Azure Cosmos DB DATALAKE: Azure Data Lake Storage POSTGRES: PostgreSQL database

BLOB = 'blob'#

COSMOS = 'cosmos'#

DATALAKE = 'datalake'#

LOCAL = 'local'#

POSTGRES = 'postgres'#

Configuration

Contents

Configuration#

Key Classes#

Environment Variables#