Evaluation Guide

Evaluate trained robotics policies using local environments, Azure ML compute, or NVIDIA OSMO workflows. This guide covers LeRobot ACT policy evaluation and OSMO-managed evaluation for Isaac Lab and LeRobot workloads.

📖 Evaluation Guides

Guide	Description
LeRobot ACT Policy Evaluation	Run LeRobot ACT policies locally with ROS2 deployment
OSMO Evaluation Workflows	Execute Isaac Lab and LeRobot evaluation via NVIDIA OSMO

⚖️ Evaluation Comparison

Feature	Local / Azure ML	OSMO
Orchestration	Manual or Azure ML jobs	OSMO workflow engine
Checkpoint source	MLflow, HuggingFace	MLflow, Azure Blob, HTTP(S)
Supported frameworks	LeRobot	Isaac Lab, LeRobot
GPU management	User-managed	KAI Scheduler
Monitoring	Local logs	osmo workflow logs

🚀 Quick Start

LeRobot local evaluation:

python lerobot/scripts/eval.py \
  --policy.path=<path-to-checkpoint> \
  -p lerobot/configs/policy/act.yaml

OSMO evaluation submission:

osmo workflow submit \
  --file evaluation/sil/workflows/osmo/eval.yaml \
  --set checkpoint_uri=<checkpoint-uri>

📚 Related Documentation

• Training Guide
• MLflow Integration
• Workflow Templates
• Scripts Reference

---

🤖 Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.