autogen_ext.models.anthropic#
- class AnthropicChatCompletionClient(**kwargs: Unpack)[source]#
Bases:
BaseAnthropicChatCompletionClient,Component[AnthropicClientConfigurationConfigModel]Chat completion client for Anthropic’s Claude models.
- Parameters:
model (str) – The Claude model to use (e.g., “claude-3-sonnet-20240229”, “claude-3-opus-20240229”)
api_key (str, optional) – Anthropic API key. Required if not in environment variables.
base_url (str, optional) – Override the default API endpoint.
max_tokens (int, optional) – Maximum tokens in the response. Default is 4096.
temperature (float, optional) – Controls randomness. Lower is more deterministic. Default is 1.0.
top_p (float, optional) – Controls diversity via nucleus sampling. Default is 1.0.
top_k (int, optional) – Controls diversity via top-k sampling. Default is -1 (disabled).
model_info (ModelInfo, optional) – The capabilities of the model. Required if using a custom model.
To use this client, you must install the Anthropic extension:
pip install "autogen-ext[anthropic]"
Example:
import asyncio from autogen_ext.models.anthropic import AnthropicChatCompletionClient from autogen_core.models import UserMessage async def main(): anthropic_client = AnthropicChatCompletionClient( model="claude-3-sonnet-20240229", api_key="your-api-key", # Optional if ANTHROPIC_API_KEY is set in environment ) result = await anthropic_client.create([UserMessage(content="What is the capital of France?", source="user")]) # type: ignore print(result) if __name__ == "__main__": asyncio.run(main())
To load the client from a configuration:
from autogen_core.models import ChatCompletionClient config = { "provider": "AnthropicChatCompletionClient", "config": {"model": "claude-3-sonnet-20240229"}, } client = ChatCompletionClient.load_component(config)
- component_type: ClassVar[ComponentType] = 'model'#
The logical type of the component.
- component_config_schema#
- component_provider_override: ClassVar[str | None] = 'autogen_ext.models.anthropic.AnthropicChatCompletionClient'#
Override the provider string for the component. This should be used to prevent internal module names being a part of the module name.
- _to_config() AnthropicClientConfigurationConfigModel[source]#
Dump the configuration that would be requite to create a new instance of a component matching the configuration of this instance.
- Returns:
T – The configuration of the component.
- classmethod _from_config(config: AnthropicClientConfigurationConfigModel) Self[source]#
Create a new instance of the component from a configuration object.
- Parameters:
config (T) – The configuration object.
- Returns:
Self – The new instance of the component.
- class AnthropicBedrockChatCompletionClient(**kwargs: Unpack)[source]#
Bases:
BaseAnthropicChatCompletionClient,Component[AnthropicBedrockClientConfigurationConfigModel]Chat completion client for Anthropic’s Claude models on AWS Bedrock.
- Parameters:
model (str) – The Claude model to use (e.g., “claude-3-sonnet-20240229”, “claude-3-opus-20240229”)
api_key (str, optional) – Anthropic API key. Required if not in environment variables.
base_url (str, optional) – Override the default API endpoint.
max_tokens (int, optional) – Maximum tokens in the response. Default is 4096.
temperature (float, optional) – Controls randomness. Lower is more deterministic. Default is 1.0.
top_p (float, optional) – Controls diversity via nucleus sampling. Default is 1.0.
top_k (int, optional) – Controls diversity via top-k sampling. Default is -1 (disabled).
model_info (ModelInfo, optional) – The capabilities of the model. Required if using a custom model.
bedrock_info (BedrockInfo, optional) – The capabilities of the model in bedrock. Required if using a model from AWS bedrock.
To use this client, you must install the Anthropic extension:
pip install "autogen-ext[anthropic]"
Example:
import asyncio from autogen_ext.models.anthropic import AnthropicBedrockChatCompletionClient, BedrockInfo from autogen_core.models import UserMessage, ModelInfo async def main(): anthropic_client = AnthropicBedrockChatCompletionClient( model="anthropic.claude-3-5-sonnet-20240620-v1:0", temperature=0.1, model_info=ModelInfo( vision=False, function_calling=True, json_output=False, family="unknown", structured_output=True ), bedrock_info=BedrockInfo( aws_access_key="<aws_access_key>", aws_secret_key="<aws_secret_key>", aws_session_token="<aws_session_token>", aws_region="<aws_region>", ), ) result = await anthropic_client.create([UserMessage(content="What is the capital of France?", source="user")]) # type: ignore print(result) if __name__ == "__main__": asyncio.run(main())
- component_type: ClassVar[ComponentType] = 'model'#
The logical type of the component.
- component_config_schema#
- component_provider_override: ClassVar[str | None] = 'autogen_ext.models.anthropic.AnthropicBedrockChatCompletionClient'#
Override the provider string for the component. This should be used to prevent internal module names being a part of the module name.
- _to_config() AnthropicBedrockClientConfigurationConfigModel[source]#
Dump the configuration that would be requite to create a new instance of a component matching the configuration of this instance.
- Returns:
T – The configuration of the component.
- classmethod _from_config(config: AnthropicBedrockClientConfigurationConfigModel) Self[source]#
Create a new instance of the component from a configuration object.
- Parameters:
config (T) – The configuration object.
- Returns:
Self – The new instance of the component.
- class BaseAnthropicChatCompletionClient(client: Any, *, create_args: Dict[str, Any], model_info: ModelInfo | None = None)[source]#
Bases:
ChatCompletionClient- async create(messages: Sequence[Annotated[SystemMessage | UserMessage | AssistantMessage | FunctionExecutionResultMessage, FieldInfo(annotation=NoneType, required=True, discriminator='type')]], *, tools: Sequence[Tool | ToolSchema] = [], tool_choice: Tool | Literal['auto', 'required', 'none'] = 'auto', json_output: bool | type[BaseModel] | None = None, extra_create_args: Mapping[str, Any] = {}, cancellation_token: CancellationToken | None = None) CreateResult[source]#
Creates a single response from the model.
- Parameters:
messages (Sequence[LLMMessage]) – The messages to send to the model.
tools (Sequence[Tool | ToolSchema], optional) – The tools to use with the model. Defaults to [].
tool_choice (Tool | Literal["auto", "required", "none"], optional) – A single Tool object to force the model to use, “auto” to let the model choose any available tool, “required” to force tool usage, or “none” to disable tool usage. Defaults to “auto”.
json_output (Optional[bool | type[BaseModel]], optional) – Whether to use JSON mode, structured output, or neither. Defaults to None. If set to a Pydantic BaseModel type, it will be used as the output type for structured output. If set to a boolean, it will be used to determine whether to use JSON mode or not. If set to True, make sure to instruct the model to produce JSON output in the instruction or prompt.
extra_create_args (Mapping[str, Any], optional) – Extra arguments to pass to the underlying client. Defaults to {}.
cancellation_token (Optional[CancellationToken], optional) – A token for cancellation. Defaults to None.
- Returns:
CreateResult – The result of the model call.
- async create_stream(messages: Sequence[Annotated[SystemMessage | UserMessage | AssistantMessage | FunctionExecutionResultMessage, FieldInfo(annotation=NoneType, required=True, discriminator='type')]], *, tools: Sequence[Tool | ToolSchema] = [], tool_choice: Tool | Literal['auto', 'required', 'none'] = 'auto', json_output: bool | type[BaseModel] | None = None, extra_create_args: Mapping[str, Any] = {}, cancellation_token: CancellationToken | None = None, max_consecutive_empty_chunk_tolerance: int = 0) AsyncGenerator[str | CreateResult, None][source]#
Creates an AsyncGenerator that yields a stream of completions based on the provided messages and tools.
- count_tokens(messages: Sequence[Annotated[SystemMessage | UserMessage | AssistantMessage | FunctionExecutionResultMessage, FieldInfo(annotation=NoneType, required=True, discriminator='type')]], *, tools: Sequence[Tool | ToolSchema] = []) int[source]#
Estimate the number of tokens used by messages and tools.
Note: This is an estimation based on common tokenization patterns and may not perfectly match Anthropic’s exact token counting for Claude models.
- remaining_tokens(messages: Sequence[Annotated[SystemMessage | UserMessage | AssistantMessage | FunctionExecutionResultMessage, FieldInfo(annotation=NoneType, required=True, discriminator='type')]], *, tools: Sequence[Tool | ToolSchema] = []) int[source]#
Calculate the remaining tokens based on the model’s token limit.
- actual_usage() RequestUsage[source]#
- total_usage() RequestUsage[source]#
- property capabilities: ModelCapabilities#
- class AnthropicClientConfiguration[source]#
Bases:
BaseAnthropicClientConfiguration- response_format: ResponseFormat | None#
- model_capabilities: ModelCapabilities#
- class AnthropicBedrockClientConfiguration[source]#
Bases:
AnthropicClientConfiguration- bedrock_info: BedrockInfo#
- response_format: ResponseFormat | None#
- model_capabilities: ModelCapabilities#
- pydantic model AnthropicClientConfigurationConfigModel[source]#
Bases:
BaseAnthropicClientConfigurationConfigModelShow JSON schema
{ "title": "AnthropicClientConfigurationConfigModel", "type": "object", "properties": { "model": { "title": "Model", "type": "string" }, "max_tokens": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": 4096, "title": "Max Tokens" }, "temperature": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": 1.0, "title": "Temperature" }, "top_p": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Top P" }, "top_k": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Top K" }, "stop_sequences": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Stop Sequences" }, "response_format": { "anyOf": [ { "$ref": "#/$defs/ResponseFormat" }, { "type": "null" } ], "default": null }, "metadata": { "anyOf": [ { "additionalProperties": { "type": "string" }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Metadata" }, "api_key": { "anyOf": [ { "format": "password", "type": "string", "writeOnly": true }, { "type": "null" } ], "default": null, "title": "Api Key" }, "base_url": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Base Url" }, "model_capabilities": { "anyOf": [ { "$ref": "#/$defs/ModelCapabilities" }, { "type": "null" } ], "default": null }, "model_info": { "anyOf": [ { "$ref": "#/$defs/ModelInfo" }, { "type": "null" } ], "default": null }, "timeout": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Timeout" }, "max_retries": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Max Retries" }, "default_headers": { "anyOf": [ { "additionalProperties": { "type": "string" }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Default Headers" }, "tools": { "anyOf": [ { "items": { "type": "object" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Tools" }, "tool_choice": { "anyOf": [ { "enum": [ "auto", "any", "none" ], "type": "string" }, { "type": "object" }, { "type": "null" } ], "default": null, "title": "Tool Choice" } }, "$defs": { "ModelCapabilities": { "deprecated": true, "properties": { "vision": { "title": "Vision", "type": "boolean" }, "function_calling": { "title": "Function Calling", "type": "boolean" }, "json_output": { "title": "Json Output", "type": "boolean" } }, "required": [ "vision", "function_calling", "json_output" ], "title": "ModelCapabilities", "type": "object" }, "ModelInfo": { "description": "ModelInfo is a dictionary that contains information about a model's properties.\nIt is expected to be used in the model_info property of a model client.\n\nWe are expecting this to grow over time as we add more features.", "properties": { "vision": { "title": "Vision", "type": "boolean" }, "function_calling": { "title": "Function Calling", "type": "boolean" }, "json_output": { "title": "Json Output", "type": "boolean" }, "family": { "anyOf": [ { "enum": [ "gpt-41", "gpt-45", "gpt-4o", "o1", "o3", "o4", "gpt-4", "gpt-35", "r1", "gemini-1.5-flash", "gemini-1.5-pro", "gemini-2.0-flash", "gemini-2.5-pro", "gemini-2.5-flash", "claude-3-haiku", "claude-3-sonnet", "claude-3-opus", "claude-3-5-haiku", "claude-3-5-sonnet", "claude-3-7-sonnet", "claude-4-opus", "claude-4-sonnet", "llama-3.3-8b", "llama-3.3-70b", "llama-4-scout", "llama-4-maverick", "codestral", "open-codestral-mamba", "mistral", "ministral", "pixtral", "unknown" ], "type": "string" }, { "type": "string" } ], "title": "Family" }, "structured_output": { "title": "Structured Output", "type": "boolean" }, "multiple_system_messages": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "title": "Multiple System Messages" } }, "required": [ "vision", "function_calling", "json_output", "family", "structured_output" ], "title": "ModelInfo", "type": "object" }, "ResponseFormat": { "properties": { "type": { "enum": [ "text", "json_object" ], "title": "Type", "type": "string" } }, "required": [ "type" ], "title": "ResponseFormat", "type": "object" } }, "required": [ "model" ] }
- Fields:
- pydantic model AnthropicBedrockClientConfigurationConfigModel[source]#
Bases:
AnthropicClientConfigurationConfigModelShow JSON schema
{ "title": "AnthropicBedrockClientConfigurationConfigModel", "type": "object", "properties": { "model": { "title": "Model", "type": "string" }, "max_tokens": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": 4096, "title": "Max Tokens" }, "temperature": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": 1.0, "title": "Temperature" }, "top_p": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Top P" }, "top_k": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Top K" }, "stop_sequences": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Stop Sequences" }, "response_format": { "anyOf": [ { "$ref": "#/$defs/ResponseFormat" }, { "type": "null" } ], "default": null }, "metadata": { "anyOf": [ { "additionalProperties": { "type": "string" }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Metadata" }, "api_key": { "anyOf": [ { "format": "password", "type": "string", "writeOnly": true }, { "type": "null" } ], "default": null, "title": "Api Key" }, "base_url": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Base Url" }, "model_capabilities": { "anyOf": [ { "$ref": "#/$defs/ModelCapabilities" }, { "type": "null" } ], "default": null }, "model_info": { "anyOf": [ { "$ref": "#/$defs/ModelInfo" }, { "type": "null" } ], "default": null }, "timeout": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Timeout" }, "max_retries": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Max Retries" }, "default_headers": { "anyOf": [ { "additionalProperties": { "type": "string" }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Default Headers" }, "tools": { "anyOf": [ { "items": { "type": "object" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Tools" }, "tool_choice": { "anyOf": [ { "enum": [ "auto", "any", "none" ], "type": "string" }, { "type": "object" }, { "type": "null" } ], "default": null, "title": "Tool Choice" }, "bedrock_info": { "anyOf": [ { "$ref": "#/$defs/BedrockInfoConfigModel" }, { "type": "null" } ], "default": null } }, "$defs": { "BedrockInfoConfigModel": { "properties": { "aws_access_key": { "format": "password", "title": "Aws Access Key", "type": "string", "writeOnly": true }, "aws_session_token": { "format": "password", "title": "Aws Session Token", "type": "string", "writeOnly": true }, "aws_region": { "title": "Aws Region", "type": "string" }, "aws_secret_key": { "format": "password", "title": "Aws Secret Key", "type": "string", "writeOnly": true } }, "required": [ "aws_access_key", "aws_session_token", "aws_region", "aws_secret_key" ], "title": "BedrockInfoConfigModel", "type": "object" }, "ModelCapabilities": { "deprecated": true, "properties": { "vision": { "title": "Vision", "type": "boolean" }, "function_calling": { "title": "Function Calling", "type": "boolean" }, "json_output": { "title": "Json Output", "type": "boolean" } }, "required": [ "vision", "function_calling", "json_output" ], "title": "ModelCapabilities", "type": "object" }, "ModelInfo": { "description": "ModelInfo is a dictionary that contains information about a model's properties.\nIt is expected to be used in the model_info property of a model client.\n\nWe are expecting this to grow over time as we add more features.", "properties": { "vision": { "title": "Vision", "type": "boolean" }, "function_calling": { "title": "Function Calling", "type": "boolean" }, "json_output": { "title": "Json Output", "type": "boolean" }, "family": { "anyOf": [ { "enum": [ "gpt-41", "gpt-45", "gpt-4o", "o1", "o3", "o4", "gpt-4", "gpt-35", "r1", "gemini-1.5-flash", "gemini-1.5-pro", "gemini-2.0-flash", "gemini-2.5-pro", "gemini-2.5-flash", "claude-3-haiku", "claude-3-sonnet", "claude-3-opus", "claude-3-5-haiku", "claude-3-5-sonnet", "claude-3-7-sonnet", "claude-4-opus", "claude-4-sonnet", "llama-3.3-8b", "llama-3.3-70b", "llama-4-scout", "llama-4-maverick", "codestral", "open-codestral-mamba", "mistral", "ministral", "pixtral", "unknown" ], "type": "string" }, { "type": "string" } ], "title": "Family" }, "structured_output": { "title": "Structured Output", "type": "boolean" }, "multiple_system_messages": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "title": "Multiple System Messages" } }, "required": [ "vision", "function_calling", "json_output", "family", "structured_output" ], "title": "ModelInfo", "type": "object" }, "ResponseFormat": { "properties": { "type": { "enum": [ "text", "json_object" ], "title": "Type", "type": "string" } }, "required": [ "type" ], "title": "ResponseFormat", "type": "object" } }, "required": [ "model" ] }
- field bedrock_info: BedrockInfoConfigModel | None = None#
- pydantic model CreateArgumentsConfigModel[source]#
Bases:
BaseModelShow JSON schema
{ "title": "CreateArgumentsConfigModel", "type": "object", "properties": { "model": { "title": "Model", "type": "string" }, "max_tokens": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": 4096, "title": "Max Tokens" }, "temperature": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": 1.0, "title": "Temperature" }, "top_p": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Top P" }, "top_k": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Top K" }, "stop_sequences": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "Stop Sequences" }, "response_format": { "anyOf": [ { "$ref": "#/$defs/ResponseFormat" }, { "type": "null" } ], "default": null }, "metadata": { "anyOf": [ { "additionalProperties": { "type": "string" }, "type": "object" }, { "type": "null" } ], "default": null, "title": "Metadata" } }, "$defs": { "ResponseFormat": { "properties": { "type": { "enum": [ "text", "json_object" ], "title": "Type", "type": "string" } }, "required": [ "type" ], "title": "ResponseFormat", "type": "object" } }, "required": [ "model" ] }
- Fields:
- field response_format: ResponseFormat | None = None#
- class BedrockInfo[source]#
Bases:
TypedDictBedrockInfo is a dictionary that contains information about a bedrock’s properties. It is expected to be used in the bedrock_info property of a model client.