Evaluation Guide

Evaluate trained robotics policies using local environments, Azure ML compute, or NVIDIA OSMO workflows. This guide covers LeRobot ACT policy evaluation and OSMO-managed evaluation for Isaac Lab and LeRobot workloads.

📖 Evaluation Guides

Guide	Description
LeRobot ACT Policy Evaluation	Run LeRobot ACT policies locally with ROS2 deployment
OSMO Evaluation Workflows	Execute Isaac Lab and LeRobot evaluation via NVIDIA OSMO
HiL Evaluation	Run CPU and independently no-command HiL gates

⚖️ Evaluation Comparison

Feature	Local / Azure ML	OSMO
Orchestration	Manual or Azure ML jobs	OSMO workflow engine
Checkpoint source	MLflow, HuggingFace	MLflow, Azure Blob, HTTP(S)
Supported frameworks	LeRobot	Isaac Lab, LeRobot
GPU management	User-managed	KAI Scheduler
Monitoring	Local logs	osmo workflow logs

🚀 Quick Start

LeRobot local evaluation:

python lerobot/scripts/eval.py \
  --policy.path=<path-to-checkpoint> \
  -p lerobot/configs/policy/act.yaml

OSMO evaluation submission:

osmo workflow submit \
  --file evaluation/sil/workflows/osmo/eval.yaml \
  --set checkpoint_uri=<checkpoint-uri>

📚 Related Documentation

• Training Guide
• MLflow Integration
• Workflow Templates
• Scripts Reference

---

🤖 Crafted with precision by ✨Copilot following brilliant human instruction, then carefully refined by our team of discerning human reviewers.