Getting started with flex flow using Unify AI#

Authored by:  AvatarOpen on GitHub

Learning Objectives - Upon completing this tutorial, you should be able to:

  • Write LLM application using Unify AI API in notebook and visualize the trace of your application.

  • Choose a model/provider from Unify model catalogue for your flex flow.

  • Convert the application into a flow and batch run against multi lines of data.

0. Install dependent packages#

%%capture --no-stderr
%pip install -r ./requirements.txt

1. Trace your application with promptflow#

Assume we already have a python function that calls Unify AI.

with open("llm.py") as fin:
    print(fin.read())

Note: before running below cell, please configure required environment variable UNIFY_AI_API_KEY by create an .env file. Please refer to ./.env.example as an template.

Choose LLM model and provider#

Define provider and model from Unify AI. Refer to Unify model catalogue

Choose an optimal model/provider combination for your usecase by comparing trade-offs between quality, cost and latency.

# model_name and provider_name defined here are used throughout the notebook.
# This example use llama 3.1 8b params and together-ai, redefine as per your usecase.

model_name = "llama-3.1-8b-chat"
provider_name = "together-ai"

from llm import my_llm_tool

# pls configure `UNIFY_AI_API_KEY` environment variable
result = my_llm_tool(
    prompt="Write a simple Hello, world! python program that displays the greeting message when executed. Output code only.",
    model_name=model_name,
    provider_name=provider_name,
)
result

Visualize trace by using start_trace#

Note we add @trace in the my_llm_tool function, re-run below cell will collect a trace in trace UI.

from promptflow.tracing import start_trace

# start a trace session, and print a url for user to check trace
start_trace()
# rerun the function, which will be recorded in the trace
result = my_llm_tool(
    prompt="Write a simple Hello, world! program that displays the greeting message when executed. Output code only.",
    model_name=model_name,
    provider_name=provider_name,
)
result

Now, let’s add another layer of function call. In programmer.py there is a function called write_simple_program, which calls a new function called load_prompt and previous my_llm_tool function.

# show the programmer.py content
with open("programmer.py") as fin:
    print(fin.read())

# call the flow entry function
from programmer import write_simple_program

result = write_simple_program("Java Hello, world!")
result

Setup model configuration with environment variables#

When used in local, create a model configuration object with environment variables.

Note: before running below cell, please configure required environment variable UNIFY_AI_API_KEY and UNIFY_AI_BASE_URL by creating a .env file. Please refer to ./.env.example as an template.

Here OpenAI client is being used to call Unify AI API. UNIFY_AI_BASE_URL is the base url for the API endpoint (along with version) in Unify API Documentation. ./.env.example contains base url for for Unify API version v0.

import os
from dotenv import load_dotenv

from promptflow.core import OpenAIModelConfiguration

# pls configure `UNIFY_AI_API_KEY`, `UNIFY_AI_BASE_URL` environment variables first
if "UNIFY_AI_API_KEY" not in os.environ:
    # load environment variables from .env file
    load_dotenv()

if "UNIFY_AI_API_KEY" not in os.environ:
    raise Exception("Please specify environment variables: UNIFY_AI_API_KEY")
model_config = OpenAIModelConfiguration(
    base_url=os.environ["UNIFY_AI_BASE_URL"],
    api_key=os.environ["UNIFY_AI_API_KEY"],
    model=f"{model_name}@{provider_name}" 
)

Eval the result#

%load_ext autoreload
%autoreload 2

from eval_code_flow.code_quality_unify_ai import CodeEvaluator


evaluator = CodeEvaluator(model_config=model_config)
eval_result = evaluator(result)
eval_result

2. Batch run the function as flow with multi-line data#

Create a flow.flex.yaml file to define a flow which entry pointing to the python function we defined.

# show the flow.flex.yaml content
with open("flow.flex.yaml") as fin:
    print(fin.read())

Batch run with a data file (with multiple lines of test data)#

from promptflow.client import PFClient

pf = PFClient()

data = "./data.jsonl"  # path to the data file
# create run with the flow function and data
base_run = pf.run(
    flow=write_simple_program,
    data=data,
    column_mapping={
        "text": "${data.text}",
    },
    stream=True,
)

details = pf.get_details(base_run)
details.head(10)

3. Evaluate your flow#

Then you can use an evaluation method to evaluate your flow. The evaluation methods are also flows which usually using LLM assert the produced output matches certain expectation.

Run evaluation on the previous batch run#

The base_run is the batch run we completed in step 2 above, for web-classification flow with “data.jsonl” as input.

# we can also run flow pointing to code evaluator yaml file
eval_flow = "./eval_code_flow/code-eval-flow.flex.yaml"

eval_run = pf.run(
    flow=eval_flow,
    init={"model_config": model_config},
    data="./data.jsonl",  # path to the data file
    run=base_run,  # specify base_run as the run you want to evaluate
    column_mapping={
        "code": "${run.outputs.output}",
    },
    stream=True,
)

details = pf.get_details(eval_run)
details.head(10)

import json

metrics = pf.get_metrics(eval_run)
print(json.dumps(metrics, indent=4))

pf.visualize([base_run, eval_run])

End Note#

By now you’ve successfully run your simple code generation and evaluation using Unify AI.