Skip to main content

Updating External Components

This guide covers identification, updating, vetting, and breaking change handling for all reused externally-maintained components. It satisfies the OpenSSF Best Practices Silver documentation_reuse_component_update criterion.

For quick dependency commands, see the Component Updates section of the Pull Request Process guide. For CVE-driven security updates, see Security Review.

Component Inventory

ComponentSourceVersion LocationCurrent VersionUpdate Method
NVIDIA GPU OperatorHelminfrastructure/setup/defaults.confGPU_OPERATOR_VERSIONv26.3.2Manual
KAI SchedulerHelminfrastructure/setup/defaults.confKAI_SCHEDULER_VERSIONv0.20.1Manual
OSMO ChartHelminfrastructure/setup/defaults.confOSMO_CHART_VERSION1.3.0Manual
OSMO ImageContainerinfrastructure/setup/defaults.confOSMO_IMAGE_VERSION6.3.0Manual
AzureML K8s ExtensionAzure CLI02-deploy-azureml-extension.sh--release-train stableLatest stableAutomatic
Isaac LabContainerHardcoded in 7+ files2.3.2Manual grep
ORASBinaryscripts/security/tool-checksums.json1.2.0Manual
Azure Terraform ProvidersTerraformversions.tf across 8 directoriesFloor-pinnedDependabot (2/4)
Python Packagesuvpyproject.toml, uv.lockMixedDependabot
GitHub ActionsGitHubWorkflow YAML (18 files)SHA-pinnedDependabot

[!IMPORTANT] Isaac Lab version 2.3.2 is hardcoded across workflow YAMLs, deploy scripts, and pyproject.toml files. No centralized variable exists. Use grep -r "2.3.2" --include="*.yaml" --include="*.yml" --include="*.toml" --include="*.sh" to locate all references before updating.

Identifying Available Updates

EcosystemTool or MethodCommand or Location
PythonDependabot PRs, uv lock --upgrade.github/dependabot.yml, pyproject.toml
Shell DownloadsManual check, scripts/security/tool-checksums.jsontool-checksums.json
TerraformDependabot PRs, terraform init -upgrade.github/dependabot.yml, infrastructure/terraform/
Helm Chartshelm repo update && helm search repo <chart> --versionsNVIDIA NGC Helm repositories
Container ImagesNVIDIA NGC catalog, GitHub release pagesnvcr.io/nvidia/ namespace
GitHub ActionsDependabot PRs, gh api repos/{owner}/{repo}/releases/latest.github/dependabot.yml

Automated Updates (Dependabot)

Dependabot opens PRs weekly on Monday for covered ecosystems. Configuration lives in .github/dependabot.yml.

EcosystemDirectoryGroupingSchedule
pip/python-dependenciesWeekly, Monday
pip/training/training-dependenciesWeekly, Monday
terraform/infrastructure/terraformNoneWeekly, Monday
terraform/infrastructure/terraform/dnsNoneWeekly, Monday
github-actions/github-actionsWeekly, Monday

PR flow: Dependabot opens PR → CI runs (dependency-review, pinning-scan, CodeQL, linters) → advisory reviewer agent posts a GHSA/OSV-enriched risk summary → maintainer reviews changelog and test results → merge.

[!NOTE] Dependabot does not cover Helm charts, container images, or 2 additional Terraform directories (vpn/, automation/). These require manual updates.

Advisory Reviewer Agent

An agentic workflow at .github/workflows/aw-dependabot-pr-review.md triggers on every Dependabot PR and posts a single review with the verdict APPROVE or COMMENT. It never emits REQUEST_CHANGES and never blocks a merge.

The reviewer enriches each update with:

  • GHSA, OSV, and NVD advisory lookups for referenced CVE/GHSA IDs
  • Release-notes highlights pulled from the ecosystem registry (npm, PyPI, Go proxy, Terraform registry, Docker Hub)
  • Surface-specific risk flags (Isaac Sim numpy ABI pin, azurerm major bumps, CUDA-adjacent Docker base images, unpinned Action tags)

The review body prepends a ⚠️ Maintainer review recommended banner when any high-risk signal fires. Up to five inline comments are anchored to the changed manifest or lockfile lines. The workflow skips drafts and any PR that touches .github/workflows/**. The persona is defined in .github/agents/dependabot-pr-reviewer.agent.md.

Maintainers remain the source of truth — the reviewer is advisory context, not automated policy.

Python Lockfiles

Every Python subproject carries a committed uv.lock beside its pyproject.toml. The lock is the single resolution source of truth — runtime-flat requirements.txt files are not committed.

  • Regenerate a lock with uv lock (or uv lock --upgrade) after editing pyproject.toml. Never hand-edit uv.lock and never run uv pip compile to produce a committed flat file.
  • Derive runtime dependencies at build or submit time via uv export --frozen --no-hashes --no-emit-project piped into uv pip install --no-deps. --frozen reads the lock without regenerating it. The OSMO replay mirror (training/utils/replay-azureml.sh) derives its requirements this way from workflows/osmo/uv.lock.
  • Constrain the universal lock to supported platforms with [tool.uv] environments (for example linux x86_64 for GPU and Isaac subprojects). Preserve these markers when regenerating.
  • Dependabot regenerates affected locks natively on dependency PRs. The read-only uv lock --check gate (see CI Validation for Dependency PRs) fails any PR whose lock drifts from its manifest, so no manual uv lock step is required on Dependabot PRs.

Tool Checksums

The scripts/security/tool-checksums.json file is the repository's single source of truth for explicit tool versions and their SHA-256 digests. This file currently manages:

  • ORAS: Fetched inside the GR00T training container to push checkpoints to ACR.
  • Actionlint: Used in the devcontainer for GitHub Actions workflow linting.
  • Gitleaks: Used in the devcontainer for secret scanning.

Updating Tool Checksums

When you need to update a tool managed by this manifest (e.g. bumping ORAS to a new release):

  1. Find the target version and release asset: Visit the tool's upstream release page (e.g. https://github.com/oras-project/oras/releases).
  2. Retrieve the SHA-256 checksum: Download the target asset (oras_..._linux_amd64.tar.gz) or its checksum file, and run shasum -a 256 <file>.
  3. Update the manifest: Edit scripts/security/tool-checksums.json, updating the version and sha256 fields for the appropriate entry.
  4. Commit the change: All downstream consumers dynamically read from this file at runtime; no secondary edits are required.

[!WARNING] Do not attempt to update these tools directly in shell scripts. The CI pinning scanner will flag mismatches if download URLs point to one version while checking against another, but the canonical version and hash live in tool-checksums.json.

Manual Update Process

Helm Charts

Helm chart versions are centralized in infrastructure/setup/defaults.conf.

  1. Check for a new chart version:

    helm repo update
    helm search repo <chart-name> --versions
  2. Update the version variable in infrastructure/setup/defaults.conf

  3. Run --config-preview on affected deploy scripts to verify configuration

  4. Deploy to a test cluster and validate

  5. Submit PR with changelog summary from the upstream release

Container Images (Isaac Lab)

Runtime GPU images are digest-pinned: the human-readable tag stays for legibility while an immutable @sha256:<digest> makes the pull tamper-evident. Dependabot's docker ecosystem only tracks Dockerfiles, so these tag-plus-digest references are bumped manually.

  1. Check NVIDIA NGC for a new Isaac Lab release

  2. Search for all current version references:

    grep -r "2.3.2" --include="*.yaml" --include="*.yml" --include="*.toml" --include="*.sh"
  3. Resolve the digest the new tag points to:

    docker buildx imagetools inspect nvcr.io/nvidia/isaac-lab:<version> --format '{{.Manifest.Digest}}'
  4. Update every reference to <version>@sha256:<digest>:

    • scripts/lib/common.shDEFAULT_ISAAC_LAB_IMAGE (embed <version>@sha256:<digest>; DEFAULT_ISAAC_LAB_IMAGE_VERSION is derived from it automatically)
    • the OSMO workflow fallback image: lines (kept in sync with DEFAULT_ISAAC_LAB_IMAGE)
    • setup-dev.shISAACLAB_COMMIT, the matching IsaacLab git commit cloned for intellisense
    • pyproject.toml and any remaining tag-only references
  5. The GR00T base image (pytorch/pytorch) in training/vla/workflows/osmo/groot-train.yaml is digest-pinned the same way; refresh its digest when bumping that tag.

  6. Test a training workflow with the new image

  7. Submit PR with migration notes from the NVIDIA release changelog

Terraform Providers

For directories not covered by Dependabot (vpn/, automation/):

  1. Run terraform init -upgrade in the target directory
  2. Run terraform plan -var-file=terraform.tfvars to verify no breaking changes
  3. Submit PR with provider changelog references

Vetting Criteria

Apply this checklist before merging any component update.

CriterionCheckRequired For
Changelog reviewRead release notes for breaking changesAll updates
API compatibilityVerify no breaking API changes affect current usageMajor and minor updates
License checkConfirm license unchanged or still OSI-approvedAll updates
Security advisoriesCheck GitHub Security Advisories, NVDAll updates
CI passageAll CI checks pass on the update PRAll updates
Deployment test--config-preview then deploy to test clusterHelm and container updates

Breaking Change Handling

  1. Identify breaking changes from the upstream changelog and migration guides
  2. Assess impact on deployment scripts, training workflows, and CI
  3. Create migration steps in the PR description
  4. Update affected documentation (README files, deployment guides, workflow templates)
  5. Add breaking-change label to the PR
  6. Request review from infrastructure owners (@microsoft/edge-ai-core-dev)

CI Validation for Dependency PRs

These workflows validate dependency update PRs automatically.

WorkflowPurposeScope
dependency-review.ymlBlock moderate+ vulnerabilitiesAll dependency PRs
dependency-pinning-scan.ymlEnforce 95% SHA pinning complianceGitHub Actions
codeql-analysis.ymlStatic analysis for Python codePython changes
scorecard.ymlOpenSSF Scorecard assessmentRepository-wide
uv-lock-consistency.ymlFail on uv.lock/manifest driftPython lockfiles

Security-Critical Updates

For CVE-driven updates requiring expedited handling:

  1. Maintainer identifies a CVE affecting a project dependency
  2. Open a priority PR referencing the security advisory
  3. Target 24-48 hour review turnaround
  4. Update SECURITY.md if disclosure is warranted

See Security Review for the full security update process.

🤖 Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.