Manage connections

Connection helps securely store and manage secret keys or other sensitive credentials required for interacting with LLM (Large Language Models) and other external tools, for example, Azure Content Safety.

Note

This document target for manage connections locally. To use Azure AI connections locally, refer to this guide.

Connection types

There are multiple types of connections supported in promptflow, which can be simply categorized into strong type connection and custom connection. The strong type connection includes AzureOpenAIConnection, OpenAIConnection, etc. The custom connection is a generic connection type that can be used to store custom defined credentials.

We are going to use AzureOpenAIConnection as an example for strong type connection, and CustomConnection to show how to manage connections.

Create a connection

Note

If you are using WSL or other OS without default keyring storage backend, you may encounter StoreConnectionEncryptionKeyError, please refer to FAQ for the solutions.

CLI

Each of the strong type connection has a corresponding yaml schema, the example below shows the AzureOpenAIConnection yaml:

$schema: https://azuremlschemas.azureedge.net/promptflow/latest/AzureOpenAIConnection.schema.json
name: azure_open_ai_connection
type: azure_open_ai
api_key: "<to-be-replaced>"
api_base: "https://<name>.openai.azure.com/"
api_type: "azure"
api_version: "2023-03-15-preview"

The custom connection yaml will have two dict fields for secrets and configs, the example below shows the CustomConnection yaml:

$schema: https://azuremlschemas.azureedge.net/promptflow/latest/CustomConnection.schema.json
name: custom_connection
type: custom
configs:
  endpoint: "<your-endpoint>"
  other_config: "other_value"
secrets:  # required
  my_key: "<your-api-key>"

After preparing the yaml file, use the CLI command below to create them:

# Override keys with --set to avoid yaml file changes
pf connection create -f <path-to-azure-open-ai-connection> --set api_key=<your-api-key>
# Create the custom connection
pf connection create -f <path-to-custom-connection> --set configs.endpoint=<endpoint> secrets.my_key=<your-api-key>

The expected result is as follows if the connection created successfully.

SDK

Using SDK, each connection type has a corresponding class to create a connection. The following code snippet shows how to import the required class and create the connection:

from promptflow.client import PFClient
from promptflow.entities import AzureOpenAIConnection, CustomConnection

# Get a pf client to manage connections
pf = PFClient()

# Initialize an AzureOpenAIConnection object
connection = AzureOpenAIConnection(
    name="my_azure_open_ai_connection", 
    api_key="<your-api-key>", 
    api_base="<your-endpoint>",
    api_version="2023-03-15-preview"
)

# Create the connection, note that api_key will be scrubbed in the returned result
result = pf.connections.create_or_update(connection)
print(result)

# Initialize a custom connection object
connection = CustomConnection(
    name="my_custom_connection", 
    # Secrets is a required field for custom connection
    secrets={"my_key": "<your-api-key>"},
    configs={"endpoint": "<your-endpoint>", "other_config": "other_value"}
)

# Create the connection, note that all secret values will be scrubbed in the returned result
result = pf.connections.create_or_update(connection)
print(result)

VS Code Extension

On the VS Code primary sidebar > prompt flow pane. You can find the connections pane to manage your local connections. Click the “+” icon on the top right of it and follow the popped out instructions to create your new connection.

Update a connection

CLI

The commands below show how to update existing connections with new values:

# Update an azure OpenAI connection with a new api base
pf connection update -n my_azure_open_ai_connection --set api_base='new_value'
# Update a custom connection
pf connection update -n my_custom_connection --set configs.other_config='new_value'

SDK

The code snippet below shows how to update existing connections with new values:

# Update an azure OpenAI connection with a new api base
connection = pf.connections.get(name="my_azure_open_ai_connection")
connection.api_base = "new_value"
connection.api_key = "<original-key>"  # secrets are required when updating connection using sdk
result = pf.connections.create_or_update(connection)
print(connection)
# Update a custom connection
connection = pf.connections.get(name="my_custom_connection")
connection.configs["other_config"] = "new_value"
connection.secrets = {"key1": "val1"}  # secrets are required when updating connection using sdk
result = pf.connections.create_or_update(connection)
print(connection)

VS Code Extension

On the VS Code primary sidebar > prompt flow pane. You can find the connections pane to manage your local connections. Right click the item of the connection list to update or delete your connections.

List connections

CLI

List connection command will return the connections with json list format, note that all secrets and api keys will be scrubbed:

pf connection list

SDK

List connection command will return the connections object list, note that all secrets and api keys will be scrubbed:

from promptflow.client import PFClient
# Get a pf client to manage connections
pf = PFClient()
# List and print connections
connection_list = pf.connections.list()
for connection in connection_list:
    print(connection)

VS Code Extension

Delete a connection

CLI

Delete a connection with the following command:

pf connection delete -n <connection_name>

SDK

Delete a connection with the following code snippet:

from promptflow.client import PFClient

# Get a pf client to manage connections
pf = PFClient()
# Delete the connection with specific name
pf.connections.delete(name="my_custom_connection")

VS Code Extension

On the VS Code primary sidebar > prompt flow pane. You can find the connections pane to manage your local connections. Right click the item of the connection list to update or delete your connections.

Load from environment variables

With promptflow>=1.8.0, user is able to load a connection object from os environment variables with <ConnectionType>.from_env func. Note that the connection object will NOT BE CREATED to local database.

Supported types are as follows:

Connection Type	Field	Relevant Environment Variable
OpenAIConnection	api_key	OPENAI_API_KEY
	organization	OPENAI_ORG_ID
	base_url	OPENAI_BASE_URL
AzureOpenAIConnection	api_key	AZURE_OPENAI_API_KEY
	api_base	AZURE_OPENAI_ENDPOINT
	api_version	OPENAI_API_VERSION

For example, with OPENAI_API_KEY set to environment, an OpenAIConnection object can be loaded with OpenAIConnection.from_env().

Authenticate with Microsoft Entra ID

Microsoft Entra ID is a cloud-based identity and access management service that enables your employees access external resources.

Some promptflow connection types supports connection authentication with Microsoft Entra ID.

Connection Type	Yaml Field	Value	Package Requirements	VS Code Extension
AzureOpenAIConnection	auth_mode	meid_token	promptflow[azureml-serving]>=1.7.0, promptflow-tools>=1.4.0	1.20.0

Next steps

• Reach more detail about connection concepts.

• Try the connection samples.

• Consume connections from Azure AI.