Agent Server Overview

The Agent Server is the central orchestration engine that transforms UFO into a distributed multi-agent system, enabling seamless task coordination across heterogeneous devices through persistent WebSocket connections and robust state management.

New to the Agent Server? Start with the Quick Start Guide to get up and running in minutes.

What is the Agent Server?

The Agent Server is a FastAPI-based asynchronous WebSocket server that serves as the communication hub for UFO's distributed architecture. It bridges constellation orchestrators, device agents, and external systems through a unified protocol interface.

Core Responsibilities

Why Use the Agent Server?

• Centralized Control: Single point of orchestration for multi-device workflows
• Protocol Abstraction: Clients communicate via AIP, hiding network complexity
• Async by Design: Non-blocking execution enables high concurrency
• Platform Agnostic: Supports Windows, Linux, macOS (in development)

The Agent Server is part of UFO's distributed server-client architecture, where it handles orchestration and state management while Agent Clients handle command execution. See Server-Client Architecture for the complete design rationale and communication patterns.

Architecture

The server follows a clean separation of concerns with distinct layers for web service, connection management, and protocol handling.

Architectural Overview

Component Interaction Diagram:

This layered design ensures each component has a single, well-defined responsibility. The managers maintain state while the handler implements protocol logic.

Core Components

Component Documentation: - Session Manager - Session lifecycle and background execution - Client Connection Manager - Connection registry and client tracking - WebSocket Handler - AIP protocol message handling - HTTP API - REST endpoint specifications

Key Capabilities

1. Multi-Client Coordination

The server supports two distinct client types with different roles in the distributed architecture.

Client Type Comparison:

Device Clients - Execute tasks locally on Windows/Linux machines - Report hardware specs and real-time status - Respond to commands via MCP tool servers - Stream execution logs back to server

See Agent Client Overview for detailed client architecture.

Constellation Clients
- Orchestrate multi-device workflows from a central point - Dispatch tasks to specific target devices via target_id - Coordinate complex cross-device DAG execution - Aggregate results from multiple devices

Both client types connect to /ws and register using the REGISTER message. The server differentiates behavior based on client_type field. For the complete server-client architecture and design rationale, see Server-Client Architecture.

See Quick Start for registration examples.

2. Session Lifecycle Management

Unlike stateless HTTP servers, the Agent Server maintains session state throughout task execution, enabling multi-turn interactions and result callbacks.

Session Lifecycle State Machine:

Lifecycle Stages:

Session Manager Responsibilities:

• ✅ Platform abstraction: Hides Windows/Linux differences
• ✅ Background execution: Non-blocking async task execution
• ✅ Callback routing: Delivers results via WebSocket
• ✅ Resource cleanup: Cancels tasks on disconnect
• ✅ Result caching: Stores results for HTTP retrieval

3. Resilient Communication

The server implements the Agent Interaction Protocol (AIP), providing structured, type-safe communication with automatic failure handling.

Protocol Features:

Disconnection Handling Flow:

4. Dual API Interface

The server provides two API styles to support different integration patterns: real-time WebSocket for agents and simple HTTP for external systems.

WebSocket API (AIP-based)

Purpose: Real-time bidirectional communication with agent clients

import websockets

async with websockets.connect("ws://localhost:5000/ws") as ws:
    # Register as device client
    await ws.send(json.dumps({
        "message_type": "REGISTER",
        "client_id": "windows_agent_001",
        "client_type": "device",
        "metadata": {"platform": "windows", "gpu": "NVIDIA RTX 3080"}
    }))

HTTP REST API

Purpose: Task dispatch and monitoring from external systems (HTTP clients, CI/CD, etc.)

# Dispatch task to device
curl -X POST http://localhost:5000/api/dispatch \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "my_windows_device",
    "request": "Open Notepad and type Hello World",
    "task_name": "test_task_001"
  }'

# Response: {"status": "dispatched", "session_id": "session_abc123", "task_name": "test_task_001"}

# Retrieve results
curl http://localhost:5000/api/task_result/test_task_001

See HTTP API Reference for complete endpoint documentation.

Workflow Examples

Complete Task Dispatch Flow

End-to-End HTTP WebSocket Device Execution:

The green box highlights async execution on the device side, which doesn't block the server.

Multi-Device Constellation Workflow

Constellation Client Coordinating Multiple Devices:

The server acts as a message router, forwarding tasks to target devices and routing results back to the constellation orchestrator. See Constellation Documentation for more details on multi-device orchestration.

Platform Support

The server automatically detects client platforms and creates appropriate session implementations.

Supported Platforms:

Platform Auto-Detection:

The server automatically detects the client's platform during registration. You can override this globally with the --platform flag when needed for testing or specific deployment scenarios.

python -m ufo.server.app --platform windows  # Force Windows sessions
python -m ufo.server.app --platform linux    # Force Linux sessions
python -m ufo.server.app                     # Auto-detect (default)

For more details on platform-specific implementations, see Windows Agent and Linux Agent.

Configuration

The server runs out-of-the-box with sensible defaults. Advanced configuration inherits from UFO's central config system.

Command-Line Arguments

python -m ufo.server.app [OPTIONS]

Available Options:

# Development: Local-only with debug logging
python -m ufo.server.app --local --log-level DEBUG --port 8000

# Production: External access, info logging
python -m ufo.server.app --host 0.0.0.0 --port 5000 --log-level INFO

# Testing: Force Linux sessions
python -m ufo.server.app --platform linux --port 9000

UFO Configuration Inheritance

The server uses UFO's central configuration from config_dev.yaml:

See Configuration Guide for comprehensive config documentation.

Monitoring & Operations

Health Monitoring

Monitor server status and performance using HTTP endpoints.

Health Check Endpoints:

# Server health and uptime
curl http://localhost:5000/api/health

# Response:
# {
#   "status": "healthy",
#   "online_clients": [...]
# }

# Connected clients list
curl http://localhost:5000/api/clients

# Response:
# {
#   "online_clients": ["windows_001", "linux_002", ...]
# }

For comprehensive monitoring strategies including performance metrics collection, log aggregation patterns, alert configuration, and dashboard setup, see Monitoring Guide.

Error Handling

The server handles common failure scenarios gracefully to maintain system stability.

Disconnection Handling Matrix:

Best Practices

Development Environment

Optimize your development workflow with these recommended practices.

Recommended Development Configuration:

# Isolate to localhost, enable detailed logging
python -m ufo.server.app \
  --host 127.0.0.1 \
  --port 5000 \
  --local \
  --log-level DEBUG

Development Checklist:

• Use --local flag to prevent external access
• Enable DEBUG logging for detailed traces
• Monitor logs in separate terminal: tail -f logs/ufo_server.log
• Test with single device before adding multiple clients
• Use HTTP API for quick task dispatch testing
• Verify heartbeat monitoring with client disconnection

# Terminal 1: Start server with debug logging
python -m ufo.server.app --local --log-level DEBUG

# Terminal 2: Connect device client
python -m ufo.client.client --ws --ws-server ws://127.0.0.1:5000/ws

# Terminal 3: Dispatch test task
curl -X POST http://127.0.0.1:5000/api/dispatch \
  -H "Content-Type: application/json" \
  -d '{"client_id": "windowsagent", "request": "Open Notepad", "task_name": "test_001"}'

Production Deployment

The default configuration is not production-ready. Implement these security and reliability measures.

Production Architecture:

Production Checklist:

Example Nginx Configuration:

upstream ufo_server {
    server localhost:5000;
}

server {
    listen 443 ssl;
    server_name ufo-server.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    # WebSocket endpoint
    location /ws {
        proxy_pass http://ufo_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_read_timeout 3600s;
    }

    # HTTP API
    location /api/ {
        proxy_pass http://ufo_server;
        proxy_set_header Host $host;
    }
}

Scaling Strategies

The server can scale horizontally for high-load deployments, but requires careful session management.

Scaling Patterns:

Troubleshooting

Common Issues

Issue: Clients Can't Connect

# Symptom: Connection refused
Error: WebSocket connection to 'ws://localhost:5000/ws' failed

# Diagnosis:
1. Check server is running: curl http://localhost:5000/api/health
2. Verify port: netstat -an | grep 5000
3. Check firewall: sudo ufw status

# Solution:
# Start server with correct host binding
python -m ufo.server.app --host 0.0.0.0 --port 5000

Issue: Sessions Not Executing

# Symptom: Task dispatched but no results

# Diagnosis:
1. Check server logs for errors
2. Verify client is connected: curl http://localhost:5000/api/clients
3. Check target_id matches client_id

# Solution:
# Ensure client_id in request matches registered client
curl -X POST http://localhost:5000/api/dispatch \
  -d '{"client_id": "correct_client_id", "request": "test", "task_name": "test_001"}'

Issue: Memory Leak / High Memory Usage

# Symptom: Server memory grows over time

# Diagnosis:
1. Check session cleanup in logs
2. Monitor /api/health for session count
3. Profile with memory_profiler

# Solution:
# Ensure clients send TASK_END to complete sessions
# Restart server periodically (systemd handles this)
# Implement session timeout (future feature)

Debug Mode

# Ultra-verbose debugging
python -m ufo.server.app \
  --log-level DEBUG \
  --local \
  --port 5000 2>&1 | tee debug.log

# Watch logs in real-time
tail -f debug.log | grep -E "(ERROR|WARNING|Session|WebSocket)"

Documentation Map

Explore related documentation to deepen your understanding of the Agent Server ecosystem.

Getting Started

Architecture & Components

Operations

Related Documentation

Next Steps

Follow this recommended sequence to master the Agent Server:

1. Run the Server (5 minutes) - Follow the Quick Start Guide - Verify server responds to /api/health

2. Connect a Client (10 minutes) - Use Device Client - Verify registration in server logs - Check /api/clients endpoint

3. Dispatch Tasks (15 minutes) - Use HTTP API to send tasks - Retrieve results via /api/task_result - Observe WebSocket message flow in logs

4. Understand Architecture (30 minutes) - Read Session Manager internals - Study WebSocket Handler protocol implementation - Review AIP Protocol message types

5. Deploy to Production (varies) - Set up reverse proxy (nginx) - Configure SSL/TLS - Implement monitoring - Test failover scenarios