Tune prompts using variants#

Experimental feature

This is an experimental feature, and may change at any time. Learn more.

To better understand this part, please read Quick start and Run and evaluate a flow first.

What is variant and why should we care#

In order to help users tune the prompts in a more efficient way, we introduce the concept of variants which can help you test the model’s behavior under different conditions, such as different wording, formatting, context, temperature, or top-k, compare and find the best prompt and configuration that maximizes the model’s accuracy, diversity, or coherence.

Create a run with different variant node#

In this example, we use the flow web-classification, its node summarize_text_content has two variants: variant_0 and variant_1. The difference between them is the inputs parameters:

...
nodes:
- name: summarize_text_content
  use_variants: true
...
node_variants:
  summarize_text_content:
    default_variant_id: variant_0
    variants:
      variant_0:
        node:
          type: llm
          source:
            type: code
            path: summarize_text_content.jinja2
          inputs:
            deployment_name: text-davinci-003
            max_tokens: '128'
            temperature: '0.2'
            text: ${fetch_text_content_from_url.output}
          provider: AzureOpenAI
          connection: open_ai_connection
          api: completion
          module: promptflow.tools.aoai
      variant_1:
        node:
          type: llm
          source:
            type: code
            path: summarize_text_content__variant_1.jinja2
          inputs:
            deployment_name: text-davinci-003
            max_tokens: '256'
            temperature: '0.3'
            text: ${fetch_text_content_from_url.output}
          provider: AzureOpenAI
          connection: open_ai_connection
          api: completion
          module: promptflow.tools.aoai

You can check the whole flow definition in flow.dag.yaml.

Now we will create a variant run which uses node summarize_text_content’s variant variant_1. Assuming you are in working directory <path-to-the-sample-repo>/examples/flows/standard

CLI

Note we pass --variant to specify which variant of the node should be running.

pf run create --flow web-classification --data web-classification/data.jsonl --variant '${summarize_text_content.variant_1}' --column-mapping url='${data.url}' --stream --name my_first_variant_run

SDK

from promptflow.client import PFClient

pf = PFClient()  # get a promptflow client
flow = "web-classification"
data= "web-classification/data.jsonl"

# use the variant1 of the summarize_text_content node.
variant_run = pf.run(
    flow=flow,
    data=data,
    variant="${summarize_text_content.variant_1}",  # use variant 1.
    column_mapping={"url": "${data.url}"},
)

pf.stream(variant_run)

VS Code Extension

After the variant run is created, you can evaluate the variant run with a evaluation flow, just like you evalute a standard flow run.

Next steps#

Learn more about: