Anthropic Provider¶

Integrates Anthropic's Claude models into Amplifier. Supports streaming, tool calling, extended thinking, and ChatRequest format.

Module ID¶

provider-anthropic

Installation¶

providers:
  - module: provider-anthropic
    source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main
    config:
      default_model: claude-sonnet-4-5

Configuration¶

Core Options¶

Feature Toggles¶

Rate Limit & Retry¶

Debug Options¶

Overload Fallback¶

Supported Models¶

Extended Thinking¶

Enable extended thinking for complex reasoning tasks:

providers:
  - module: provider-anthropic
    config:
      default_model: claude-sonnet-4-5

Extended thinking is enabled at the orchestrator level via extended_thinking: true config. The orchestrator then passes it to the provider:

response = await provider.complete(chat_request, extended_thinking=True)

The response will include thinking blocks showing the model's reasoning process.

When extended thinking is requested alongside tools, the provider automatically adds the interleaved-thinking-2025-05-14 beta header so thinking blocks and tool calls can appear in the same turn.

Prompt Caching¶

Anthropic's prompt caching reduces costs by 90% for repeated content. Enable it:

providers:
  - module: provider-anthropic
    config:
      enable_prompt_caching: true

The provider automatically marks eligible content for caching based on message structure.

Web Search¶

Enable native web search (Claude 4+ models only):

providers:
  - module: provider-anthropic
    config:
      enable_web_search: true

The model can then search the web directly during conversations.

Task Budget¶

Set a token budget for the entire task on Claude 4.7+ models. When task_budget_tokens is configured, the provider automatically applies the task-budgets-2026-03-13 beta header:

providers:
  - module: provider-anthropic
    config:
      task_budget_tokens: 50000  # Minimum: 20000

The task budget limits total token usage across the task via output_config.task_budget. Only supported on Claude 4.7+ models; the option is ignored on earlier models.

Beta Headers¶

Anthropic provides experimental features through beta headers. Enable these features by adding the beta_headers configuration field.

Single beta header:

providers:
  - module: provider-anthropic
    config:
      default_model: claude-sonnet-4-5
      beta_headers: "context-1m-2025-08-07"  # Enable 1M token context window

Multiple beta headers:

providers:
  - module: provider-anthropic
    config:
      default_model: claude-sonnet-4-5
      beta_headers:
        - "context-1m-2025-08-07"
        - "future-feature-header"

1M Token Context Window¶

Claude Sonnet 4.5 supports a 1M token context window when the context-1m-2025-08-07 beta header is enabled:

providers:
  - module: provider-anthropic
    config:
      default_model: claude-sonnet-4-5
      beta_headers: "context-1m-2025-08-07"
      max_tokens: 8192  # Output tokens remain separate from context window

With this configuration: - Context window: Up to 1M tokens of input (messages, tools, system prompt) - Output tokens: Controlled by max_tokens (separate from context window) - Use case: Process large codebases, extensive documentation, or long conversation histories

Notes: - Beta features are experimental and subject to change - Check Anthropic's documentation for available beta headers - Beta headers are optional - existing configurations work unchanged - Invalid beta headers will cause API errors (fail fast) - Beta header usage is logged at initialization for observability - The task-budgets-2026-03-13 beta header is automatically applied when task_budget_tokens is configured (Claude 4.7+ only)

Rate Limit Management¶

The provider tracks rate limits from response headers and pre-emptively throttles when approaching limits:

providers:
  - module: provider-anthropic
    config:
      throttle_threshold: 0.02  # Throttle at 2% capacity remaining
      throttle_delay: 1.0  # Fallback sleep when no reset timestamp available

Overload Fallback¶

When fallback_on_overload is enabled, the provider temporarily routes requests to a lower-tier model if the higher-tier model remains overloaded after fallback_retry_count attempts. The downgrade window stays active for fallback_cooldown_seconds before automatically retrying the higher model.

providers:
  - module: provider-anthropic
    config:
      fallback_on_overload: true
      fallback_retry_count: 1
      fallback_cooldown_seconds: 1800

Events: - provider:fallback_open - Emitted when a temporary downgrade window opens - provider:fallback_active - Emitted on each request served through an active downgrade window

Retry Configuration¶

providers:
  - module: provider-anthropic
    config:
      max_retries: 5
      min_retry_delay: 1.0
      max_retry_delay: 60.0
      retry_jitter: 0.2
      overloaded_delay_multiplier: 10.0

The provider disables SDK built-in retries and manages retries itself, giving full control over backoff timing, retry-after header honoring, and per-error-class delay scaling.

Backoff Formula¶

Each retry delay is computed as follows:

base_delay   = min_retry_delay × 2^(attempt - 1)
capped_delay = min(base_delay, max_retry_delay)
scaled_delay = capped_delay × delay_multiplier          # 1.0 for most errors, 10.0 for 529
final_delay  = max(scaled_delay, retry_after)            # server retry-after as floor
sleep        = final_delay ± (final_delay × jitter)      # randomised ± jitter fraction

Example: 529 Overloaded (10× multiplier, defaults)

Total wait ≈ 310s (~5 min) before the request is abandoned.

Retry Events¶

A provider:retry event is emitted before each retry sleep with the following fields:

Error Handling¶

The provider translates Anthropic SDK exceptions to kernel errors:

The provider distinguishes Cloudflare bot-challenge 403 responses (HTML body, no JSON) from real API 403s. Cloudflare challenges are treated as transient and retried; real 403s surface as AccessDeniedError.

Events: - provider:cloudflare_challenge - Emitted when a Cloudflare bot challenge is detected on a 403 response

Graceful Error Recovery¶

The provider automatically detects and repairs broken tool call sequences before sending requests to the API.

The Problem: If tool results are missing from conversation history (due to context compaction bugs, parsing errors, or state corruption), the Anthropic API rejects the entire request.

The Solution: The provider scans messages for tool calls that have no corresponding tool result. For each missing result it injects a synthetic error result before validation:

{
    "role": "user",
    "content": [{
        "type": "tool_result",
        "tool_use_id": "toolu_123",
        "content": "[SYSTEM ERROR: Tool result missing]\n\nTool: get_weather\n..."
    }]
}

The LLM receives a complete, valid message sequence, acknowledges the error, and can ask the user to retry the failed operation.

Events: - provider:tool_sequence_repaired - Emitted when one or more synthetic tool results are injected, with details of the repaired tool IDs

Debug Events¶

The provider emits events on every request and response. Enable the raw option to include the complete unmodified API payload in those events:

providers:
  - module: provider-anthropic
    config:
      raw: true  # Include raw API payload in provider events

Events: - llm:request - Request details (message counts, model, params; includes raw payload if raw: true) - llm:response - Response details (usage, stop reason; includes raw payload if raw: true)

Environment Variables¶

export ANTHROPIC_API_KEY="your-api-key-here"
export ANTHROPIC_BASE_URL="https://api.anthropic.com"  # Optional: override API endpoint

Example Configuration¶

providers:
  - module: provider-anthropic
    source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main
    config:
      api_key: ${ANTHROPIC_API_KEY}
      default_model: claude-sonnet-4-5
      max_tokens: 8192
      temperature: 1.0
      enable_prompt_caching: true
      enable_web_search: false
      max_retries: 5
      retry_jitter: true