Async Processing with the aio Accessor¶

This notebook explains .aio.responses in three steps: default usage, batch-size tuning, and parallelism tuning.

These OpenAI API examples use gpt-6-luna with explicit reasoning={"effort": "none"}. Omitting reasoning uses the model's medium default. For Azure OpenAI, replace the model name with your deployment name and use settings supported by that deployment. See the GPT-6 migration guide. Saved outputs retained from earlier runs are illustrative and are not GPT-6 measurements.

1. Setup¶

Configure authentication first, then set the default responses model.

In [ ]:

import os

import pandas as pd

import openaivec
from openaivec import pandas_ext

assert os.getenv("OPENAI_API_KEY") or os.getenv("AZURE_OPENAI_BASE_URL"), (
    "Set OPENAI_API_KEY or Azure OpenAI environment variables before running this notebook."
)

openaivec.set_responses_model("gpt-6-luna")

import os import pandas as pd import openaivec from openaivec import pandas_ext assert os.getenv("OPENAI_API_KEY") or os.getenv("AZURE_OPENAI_BASE_URL"), ( "Set OPENAI_API_KEY or Azure OpenAI environment variables before running this notebook." ) openaivec.set_responses_model("gpt-6-luna")

2. Prepare input data¶

In [2]:

df = pd.DataFrame({
    "text": [
        "This product is amazing!",
        "Terrible customer service",
        "Good value for money",
        "Not what I expected",
    ]
    * 25
})

prompt = "Analyze sentiment and classify as positive/negative/neutral"
df.head()

df = pd.DataFrame({ "text": [ "This product is amazing!", "Terrible customer service", "Good value for money", "Not what I expected", ] * 25 }) prompt = "Analyze sentiment and classify as positive/negative/neutral" df.head()

Out[2]:

	text
0	This product is amazing!
1	Terrible customer service
2	Good value for money
3	Not what I expected
4	This product is amazing!

3. Default example (no tuning)¶

Start with no extra parameters. This uses the library defaults.

In [ ]:

default_sentiments = await df["text"].aio.responses(prompt, reasoning={"effort": "none"})
default_sentiments.head()

default_sentiments = await df["text"].aio.responses(prompt, reasoning={"effort": "none"}) default_sentiments.head()

4. Batch-size example¶

batch_size controls how many inputs are grouped into one API call. Larger values reduce request overhead, while smaller values can be safer for unstable workloads.

In [ ]:

batch_size_sentiments = await df["text"].aio.responses(
    prompt,
    reasoning={"effort": "none"},
    batch_size=32,
)
batch_size_sentiments.head()

batch_size_sentiments = await df["text"].aio.responses( prompt, reasoning={"effort": "none"}, batch_size=32, ) batch_size_sentiments.head()

5. Parallelism example¶

max_concurrency controls how many batches run in parallel. Increase gradually to improve throughput while staying within API rate limits.

In [ ]:

parallel_sentiments = await df["text"].aio.responses(
    prompt,
    reasoning={"effort": "none"},
    batch_size=32,
    max_concurrency=12,
)
parallel_sentiments.head()

parallel_sentiments = await df["text"].aio.responses( prompt, reasoning={"effort": "none"}, batch_size=32, max_concurrency=12, ) parallel_sentiments.head()

6. Summary¶

Use this order when tuning .aio.responses:

1. Start with defaults (no extra parameters).
2. Set batch_size first (16 to 64 is a practical starting range).
3. Then increase max_concurrency (4 to 12) while watching rate limits.

Goal	Suggested Setting
Keep it simple	default (batch_size=None, default concurrency)
Reduce request overhead	increase batch_size
Improve throughput	increase max_concurrency gradually
Stay stable under limits	lower both values