UFO Client Overview

The UFO Client runs on target devices and serves as the execution layer of UFO's distributed agent system. It manages MCP (Model Context Protocol) servers, executes commands deterministically, and communicates with the Agent Server through the Agent Interaction Protocol (AIP).

Quick Start: Jump to the Quick Start Guide to connect your device. Make sure the Agent Server is running first.

🎯 What is the UFO Client?

The UFO Client is a stateless execution agent that:

Stateless Design Philosophy: The client focuses purely on execution. All reasoning and decision-making happens on the server, allowing independent updates to server logic and client tools, simple client architecture, intelligent orchestration of multiple clients, and resource-efficient operation.

Architecture: The UFO Client is part of UFO's distributed server-client architecture, where it handles command execution and resource access while the Agent Server handles orchestration and decision-making. See Server-Client Architecture for the complete design rationale, communication protocols, and deployment patterns.

🏗️ Architecture

The client implements a layered architecture separating communication, execution, and tool management for maximum flexibility and maintainability.

Core Components

For detailed component documentation:

• WebSocket Client - AIP protocol implementation
• UFO Client - Execution orchestration
• Computer Manager - Multi-computer management
• Device Info Provider - System profiling
• MCP Integration - MCP server management (comprehensive documentation)

🚀 Key Capabilities

1. Deterministic Command Execution

The client executes commands exactly as specified without interpretation or reasoning, ensuring predictable behavior.

Execution Flow:

Execution Guarantees: - Isolation: Each tool runs in separate thread pool - Timeouts: Configurable timeout (default: 6000 seconds/100 minutes) - Fault Tolerance: One failed tool doesn't crash entire client - Thread Safety: Concurrent tool execution supported - Error Reporting: Structured errors returned to server

2. MCP Server Management

The client manages a collection of MCP (Model Context Protocol) servers to provide diverse tool access for automation tasks. The client is responsible for registering, managing, and executing these tools, while the Agent Server handles command orchestration. See Server-Client Architecture for how MCP integration fits into the overall architecture.

MCP Server Categories:

Data Collection Servers gather information from the device:

Example Tools: get_system_info(), list_running_apps(), capture_screenshot(), get_ui_tree()

Action Servers perform actions on the device:

Example Tools: click_button(label), type_text(text), open_application(name), execute_command(cmd)

Server Types:

Example MCP Server Configuration:

mcp_servers:
  data_collection:
    - name: "system_info"
      type: "local"
      class: "SystemInfoServer"
    - name: "ui_detector"
      type: "local"
      class: "UIDetectionServer"

  action:
    - name: "gui_automation"
      type: "local"
      class: "GUIAutomationServer"
    - name: "file_ops"
      type: "remote"
      url: "http://localhost:8080/mcp"

See MCP Integration for comprehensive MCP server documentation.

3. Device Profiling

The client automatically collects and reports device information to enable the server to make intelligent task routing decisions.

Device Profile Structure:

{
  "device_id": "device_windows_001",
  "platform": "windows",
  "platform_type": "computer",
  "os_version": "10.0.22631",
  "system_info": {
    "cpu_count": 8,
    "memory_total_gb": 16.0,
    "disk_total_gb": 512.0,
    "hostname": "DESKTOP-ABC123",
    "ip_address": "192.168.1.100"
  },
  "supported_features": [
    "gui_automation",
    "cli_execution",
    "browser_control",
    "office_integration",
    "windows_apps"
  ],
  "installed_applications": [
    "Chrome",
    "Excel",
    "PowerPoint",
    "VSCode"
  ],
  "screen_resolution": "1920x1080",
  "connected_at": "2025-11-05T10:30:00Z"
}

Profile Usage on Server:

Server Uses Profile For:

See Device Info Provider for detailed profiling documentation.

4. Resilient Communication

Robust, fault-tolerant communication with the server using strongly-typed AIP messages.

Connection Lifecycle:

Connection Features:

Message Types:

See WebSocket Client and AIP Protocol for protocol details.

📋 Workflow Examples

Client Initialization & Registration

Initialization Steps:

Command Execution Flow

🖥️ Platform Support

The client supports multiple platforms with platform-specific tool implementations.

Platform Detection:

• Automatic: Detected via platform.system() on startup
• Override: Use --platform flag to specify manually
• Validation: Server validates platform matches task requirements

Platform-Specific Example:

Windows:

# Windows-specific tools
tools = [
    "open_windows_app(name='Excel')",
    "execute_powershell(script='Get-Process')",
    "read_registry(key='HKLM\\Software')"
]

Linux:

# Linux-specific tools
tools = [
    "execute_bash(command='ls -la')",
    "install_package(name='vim')",
    "control_systemd(service='nginx', action='restart')"
]

⚙️ Configuration

Command-Line Arguments

Start the UFO client with:

python -m ufo.client.client [OPTIONS]

Available Options:

Quick Start Command:

# Minimal command (default server)
python -m ufo.client.client --ws --client-id my_device

# Production command (custom server)
python -m ufo.client.client \
  --ws \
  --client-id device_production_01 \
  --ws-server ws://ufo-server.company.com:5000/ws \
  --max-retries 10 \
  --log-level INFO

UFO Configuration

The client inherits settings from config_dev.yaml:

Key Configuration Sections:

Sample Configuration:

client:
  heartbeat_interval: 30  # seconds
  command_timeout: 6000   # seconds (100 minutes)
  max_concurrent_tools: 10

mcp_servers:
  data_collection:
    - name: system_info
      type: local
      enabled: true
  action:
    - name: gui_automation
      type: local
      enabled: true
      settings:
        click_delay: 0.5
        typing_speed: 100  # chars per minute

logging:
  level: INFO
  format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
  file: "logs/client.log"

See Configuration Guide for comprehensive documentation.

⚠️ Error Handling

The client is designed to handle various failure scenarios gracefully without crashing.

Connection Failures

Connection Error Handling:

Tool Execution Failures

Protection Mechanisms:

Error Handling Example:

# Client automatically handles tool errors
try:
    result = tool.execute(args)
    return {"status": "success", "result": result}
except TimeoutError:
    return {"status": "error", "error": "Tool execution timeout"}
except Exception as e:
    return {"status": "error", "error": str(e)}

Server Disconnection

Graceful Shutdown Process:

1. Detect Disconnection - WebSocket connection lost
2. Stop Heartbeat - Terminate keep-alive loop
3. Cancel Pending Tasks - Abort in-progress commands
4. Attempt Reconnection - Use exponential backoff
5. Clean Shutdown - If max retries exceeded

✅ Best Practices

Development Best Practices

1. Use Unique Client IDs

# Bad: Generic ID
--client-id client_001

# Good: Descriptive ID
--client-id device_win_dev_john_laptop

2. Start with INFO Logging

# Development: WARNING for normal operation (default)
--log-level WARNING

# Debugging: DEBUG for troubleshooting
--log-level DEBUG

3. Test MCP Connectivity First

# Verify MCP servers are accessible before running client
from ufo.client.mcp.mcp_server_manager import MCPServerManager

manager = MCPServerManager()
# Test server creation from configuration

Production Best Practices

1. Use Descriptive Client IDs

# Include environment, location, purpose
--client-id device_windows_production_office_01
--client-id device_linux_staging_lab_02

2. Configure Automatic Restart

systemd (Linux):

[Unit]
Description=UFO Agent Client
After=network.target

[Service]
Type=simple
User=ufo
WorkingDirectory=/opt/ufo
ExecStart=/usr/bin/python3 -m ufo.client.client \
  --ws \
  --client-id device_linux_prod_01 \
  --ws-server ws://ufo-server.internal:5000/ws \
  --log-level INFO
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

PM2 (Cross-platform):

{
  "apps": [{
    "name": "ufo-client",
    "script": "python",
    "args": [
      "-m", "ufo.client.client",
      "--ws",
      "--client-id", "device_win_prod_01",
      "--ws-server", "ws://ufo-server.internal:5000/ws",
      "--log-level", "INFO"
    ],
    "cwd": "C:\\ufo",
    "restart_delay": 5000,
    "max_restarts": 10
  }]
}

3. Monitor Connection Health

# Check logs for connection status
tail -f logs/client.log | grep -E "Connected|Disconnected|ERROR"

Security Best Practices

🎓 Documentation Map

Getting Started

Component Details

Related Documentation

🔄 Client vs. Server

Understanding the clear division between client and server responsibilities is crucial for effective system design.

Responsibility Matrix:

Workflow Comparison:

Decoupled Architecture Benefits: - Independent Updates: Modify server logic without touching clients - Flexible Deployment: Run clients on any platform - Scalability: Add more clients without server changes - Maintainability: Simpler client code, easier debugging - Testability: Test client and server independently

🚀 Next Steps

1. Run Your First Client

# Follow the quick start guide
python -m ufo.client.client \
  --ws \
  --client-id my_first_device \
  --ws-server ws://localhost:5000/ws

2. Understand Registration Process

Learn how clients register with the server, device profile structure, and registration acknowledgment.

👉 Server Quick Start - Start server and connect clients

3. Explore MCP Integration

Learn about MCP servers, configure custom tools, and create your own MCP servers.

👉 MCP Integration

4. Configure for Your Environment

Customize MCP servers, adjust timeouts and retries, and configure platform-specific settings.

👉 Configuration Guide

5. Master the Protocol

Deep dive into AIP messages, understand message flow, and error handling patterns.

👉 AIP Protocol