Async Processing with the aio Accessor¶

This notebook explains .aio.responses in three steps: default usage, batch-size tuning, and parallelism tuning.

1. Setup¶

Configure authentication first, then set the default responses model.

In [6]:

import os

import pandas as pd

from openaivec import pandas_ext

assert os.getenv("OPENAI_API_KEY") or os.getenv("AZURE_OPENAI_BASE_URL"), (
    "Set OPENAI_API_KEY or Azure OpenAI environment variables before running this notebook."
)

pandas_ext.set_responses_model("gpt-5.1")

import os import pandas as pd from openaivec import pandas_ext assert os.getenv("OPENAI_API_KEY") or os.getenv("AZURE_OPENAI_BASE_URL"), ( "Set OPENAI_API_KEY or Azure OpenAI environment variables before running this notebook." ) pandas_ext.set_responses_model("gpt-5.1")

2. Prepare input data¶

In [7]:

df = pd.DataFrame({
    "text": [
        "This product is amazing!",
        "Terrible customer service",
        "Good value for money",
        "Not what I expected",
    ]
    * 25
})

prompt = "Analyze sentiment and classify as positive/negative/neutral"
df.head()

df = pd.DataFrame({ "text": [ "This product is amazing!", "Terrible customer service", "Good value for money", "Not what I expected", ] * 25 }) prompt = "Analyze sentiment and classify as positive/negative/neutral" df.head()

Out[7]:

	text
0	This product is amazing!
1	Terrible customer service
2	Good value for money
3	Not what I expected
4	This product is amazing!

3. Default example (no tuning)¶

Start with no extra parameters. This uses the library defaults.

In [8]:

default_sentiments = await df["text"].aio.responses(prompt)
default_sentiments.head()

default_sentiments = await df["text"].aio.responses(prompt) default_sentiments.head()

Processing batches:   0%|          | 0/4 [00:00<?, ?item/s]

Out[8]:

0    Sentiment: positive
1    Sentiment: negative
2    Sentiment: positive
3    Sentiment: negative
4    Sentiment: positive
Name: text, dtype: object

4. Batch-size example¶

batch_size controls how many inputs are grouped into one API call. Larger values reduce request overhead, while smaller values can be safer for unstable workloads.

In [9]:

batch_size_sentiments = await df["text"].aio.responses(
    prompt,
    batch_size=32,
)
batch_size_sentiments.head()

batch_size_sentiments = await df["text"].aio.responses( prompt, batch_size=32, ) batch_size_sentiments.head()

Processing batches:   0%|          | 0/4 [00:00<?, ?item/s]

Out[9]:

0    Sentiment: positive
1    Sentiment: negative
2    Sentiment: positive
3    Sentiment: negative
4    Sentiment: positive
Name: text, dtype: object

5. Parallelism example¶

max_concurrency controls how many batches run in parallel. Increase gradually to improve throughput while staying within API rate limits.

In [10]:

parallel_sentiments = await df["text"].aio.responses(
    prompt,
    batch_size=32,
    max_concurrency=12,
)
parallel_sentiments.head()

parallel_sentiments = await df["text"].aio.responses( prompt, batch_size=32, max_concurrency=12, ) parallel_sentiments.head()

Processing batches:   0%|          | 0/4 [00:00<?, ?item/s]

Out[10]:

0    Sentiment: positive
1    Sentiment: negative
2    Sentiment: positive
3    Sentiment: negative
4    Sentiment: positive
Name: text, dtype: object

6. Summary¶

Use this order when tuning .aio.responses:

1. Start with defaults (no extra parameters).
2. Set batch_size first (16 to 64 is a practical starting range).
3. Then increase max_concurrency (4 to 12) while watching rate limits.

Goal	Suggested Setting
Keep it simple	default (batch_size=None, default concurrency)
Reduce request overhead	increase batch_size
Improve throughput	increase max_concurrency gradually
Stay stable under limits	lower both values