Skip to content

Build and release process

Presidio leverages Azure DevOps YAML pipelines to validate, build, release and deliver presidio. The pipelines make use of templates for code reuse using YAML Schema.

Description

The following pipelines are provided and maintained as part of presidio development process:

  • PR Validation - used to validate pull requests.
    • Linting
    • Security and compliance analysis
    • Unit tests
    • E2E tests
  • CI - triggered on merge to main branch.
    • Linting
    • Security and compliance analysis
    • Unit tests
    • E2E tests
    • deploys the artifacts to an internal dev environment.
  • Release - manually triggered.
    • releases presidio official artifacts
      • pypi
      • Microsoft container registry (and docker hub)
      • GitHub
    • updates the official demo environment.

Variables used by the pipelines

CI Pipeline

  • ACR_AZURE_SUBSCRIPTION - Service connection to Azure subscription where Azure Container Registry is.
  • ACR_REGISTRY_NAME - Name of Azure Container Registry.
  • ANALYZER_DEV_APP_NAME - Name of existing App Service for Analyzer (development environment).
  • ANONYMIZER_DEV_APP_NAME - Name of existing App Service for Anonymizer (development environment).
  • IMAGE_REDACTOR_DEV_APP_NAME - Name of existing App Service for Image Redactor (development environment).
  • DEV_AZURE_SUBSCRIPTION - Service connection to Azure subscription where App Services are (development environment).
  • DEV_RESOURCE_GROUP_NAME - Name of resource group where App Services are (development environment).

Release Pipeline

  • ACR_AZURE_SUBSCRIPTION - Service connection to Azure subscription where Azure Container Registry is.
  • ACR_REGISTRY_NAME - Name of Azure Container Registry.
  • ANALYZER_PROD_APP_NAME - Name of existing App Service for Analyzer (production environment).
  • ANONYMIZER_PROD_APP_NAME - Name of existing App Service for Anonymizer (production environment).
  • PROD_AZURE_SUBSCRIPTION - Service connection to Azure subscription where App Services are (production environment).
  • PROD_RESOURCE_GROUP_NAME - Name of resource group where App Services are (production environment).

Import a pipeline to Azure Devops

  • Sign in to your Azure DevOps organization and navigate to your project.
  • In your project, navigate to the Pipelines page. Then choose the action to create a new pipeline.
  • Walk through the steps of the wizard by first selecting 'Use the classic editor, and select GitHub as the location of your source code.
  • You might be redirected to GitHub to sign in. If so, enter your GitHub credentials.
  • When the list of repositories appears, select presidio repository.
  • Point Azure Pipelines to the relevant yaml definition you'd like to import. Set the pipeline's name, the required triggers and variables and Select Save and run.
  • A new run is started. Wait for the run to finish.

PyPI Publishing with OIDC

The GitHub Actions release workflow uses OIDC (OpenID Connect) trusted publishing to PyPI, which provides enhanced security by eliminating the need to manage PyPI API tokens. This requires:

  1. PyPI Configuration: Each package (presidio_analyzer, presidio_anonymizer, etc.) must be configured on PyPI to trust the GitHub repository and workflow.
  2. GitHub Workflow: The workflow uses pypa/gh-action-pypi-publish@release/v1 with id-token: write permissions.
  3. No Secrets Required: No PyPI API tokens need to be stored as GitHub secrets.

Benefits of OIDC: - Enhanced security through short-lived tokens - No manual token management - Automatic token rotation - Audit trail of publishing activities

Note: The Azure DevOps pipeline continues to use traditional PyPI authentication with service connections.