autogen_ext.models.openai#

class AzureOpenAIChatCompletionClient(**kwargs: Unpack)[source]#

Bases: BaseOpenAIChatCompletionClient

Chat completion client for Azure OpenAI hosted models.

Parameters:

  • azure_endpoint (str) – The endpoint for the Azure model. Required for Azure models.

  • model (str) – The deployment ID for the Azure model. Required for Azure models.

  • api_version (str) – The API version to use. Required for Azure models.

  • azure_ad_token (str) – The Azure AD token to use. Provide this or azure_ad_token_provider for token-based authentication.

  • azure_ad_token_provider (Callable[[], Awaitable[str]]) – The Azure AD token provider to use. Provide this or azure_ad_token for token-based authentication.

  • model_capabilities (ModelCapabilities) – The capabilities of the model if default resolved values are not correct.

  • api_key (optional, str) – The API key to use, use this if you are using key based authentication. It is optional if you are using Azure AD token based authentication or AZURE_OPENAI_API_KEY environment variable.

  • timeout (optional, int) – The timeout for the request in seconds.

  • max_retries (optional, int) – The maximum number of retries to attempt.

To use this client, you must install the azure and openai extensions:

pip install 'autogen-ext[openai,azure]==0.4.0.dev8'

To use the client, you need to provide your deployment id, Azure Cognitive Services endpoint, api version, and model capabilities. For authentication, you can either provide an API key or an Azure Active Directory (AAD) token credential.

The following code snippet shows how to use AAD authentication. The identity used must be assigned the Cognitive Services OpenAI User role.

from autogen_ext.models.openai import AzureOpenAIChatCompletionClient
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# Create the token provider
token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")

az_model_client = AzureOpenAIChatCompletionClient(
    azure_deployment="{your-azure-deployment}",
    model="{deployed-model, such as 'gpt-4o'}",
    api_version="2024-06-01",
    azure_endpoint="https://{your-custom-endpoint}.openai.azure.com/",
    azure_ad_token_provider=token_provider,  # Optional if you choose key-based authentication.
    # api_key="sk-...", # For key-based authentication. `AZURE_OPENAI_API_KEY` environment variable can also be used instead.
)

See here for how to use the Azure client directly or for more info.

class AzureOpenAIClientConfiguration[source]#

Bases: BaseOpenAIClientConfiguration

api_key: str#
api_version: Required[str]#
azure_ad_token: str#
azure_ad_token_provider: Callable[[], str | Awaitable[str]]#
azure_deployment: str#
azure_endpoint: Required[str]#
frequency_penalty: float | None#
logit_bias: Dict[str, int] | None#
max_retries: int#
max_tokens: int | None#
model: str#
model_capabilities: ModelCapabilities#
n: int | None#
presence_penalty: float | None#
response_format: ResponseFormat#
seed: int | None#
stop: str | None | List[str]#
temperature: float | None#
timeout: float | None#
top_p: float | None#
user: str#
class OpenAIChatCompletionClient(**kwargs: Unpack)[source]#

Bases: BaseOpenAIChatCompletionClient

Chat completion client for OpenAI hosted models.

You can also use this client for OpenAI-compatible ChatCompletion endpoints. Using this client for non-OpenAI models is not tested or guaranteed.

For non-OpenAI models, please first take a look at our community extensions for additional model clients.

Parameters:

  • model (str) – The model to use. Required.

  • api_key (str) – The API key to use. Required if ‘OPENAI_API_KEY’ is not found in the environment variables.

  • timeout (optional, int) – The timeout for the request in seconds.

  • max_retries (optional, int) – The maximum number of retries to attempt.

  • organization_id (optional, str) – The organization ID to use.

  • base_url (optional, str) – The base URL to use. Required if the model is not hosted on OpenAI.

  • model_capabilities (optional, ModelCapabilities) – The capabilities of the model. Required if the model name is not a valid OpenAI model.

To use this client, you must install the openai extension:

pip install 'autogen-ext[openai]==0.4.0.dev8'

The following code snippet shows how to use the client with an OpenAI model:

from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_core.models import UserMessage

openai_client = OpenAIChatCompletionClient(
    model="gpt-4o-2024-08-06",
    # api_key="sk-...", # Optional if you have an OPENAI_API_KEY environment variable set.
)

result = await openai_client.create([UserMessage(content="What is the capital of France?", source="user")])  # type: ignore
print(result)

To use the client with a non-OpenAI model, you need to provide the base URL of the model and the model capabilities:

from autogen_ext.models.openai import OpenAIChatCompletionClient

custom_model_client = OpenAIChatCompletionClient(
    model="custom-model-name",
    base_url="https://custom-model.com/reset/of/the/path",
    api_key="placeholder",
    model_capabilities={
        "vision": True,
        "function_calling": True,
        "json_output": True,
    },
)
class OpenAIClientConfiguration[source]#

Bases: BaseOpenAIClientConfiguration

api_key: str#
base_url: str#
frequency_penalty: float | None#
logit_bias: Dict[str, int] | None#
max_retries: int#
max_tokens: int | None#
model: str#
model_capabilities: ModelCapabilities#
n: int | None#
organization: str#
presence_penalty: float | None#
response_format: ResponseFormat#
seed: int | None#
stop: str | None | List[str]#
temperature: float | None#
timeout: float | None#
top_p: float | None#
user: str#
On this page
Edit on GitHub
Show Source