Data Pipeline

Robot-to-cloud data capture pipeline for recording, compressing, and uploading robotic training episodes. This section covers recording configuration, edge device setup, and data flow from ROS 2 nodes to Azure storage.

The capture/ directory provides configuration models, schema, examples, and tests, not a recording service. Use native ROS 2 recording options separately from the repository's proposed trigger, compression, and monitoring settings.

📖 Guides

Guide	Description
ACSA Setup for ROS 2 Bag Sync	Deploy Azure Container Storage for Arc to sync ROS 2 bag files from edge clusters to Blob Storage
Chunking and Compression Configuration	Configure bag chunking thresholds and zstd compression for ROS 2 edge recording on Jetson devices

🏗️ Architecture

data-pipeline/
├── capture/
│   ├── config/          # Recording configuration and schema
│   ├── models/          # Pydantic config models
│   └── tests/           # Config model tests
├── setup/               # Arc agent setup scripts
└── specifications/      # Domain specifications