Lab 3 Intro to the Code Interpreter

Introduction

The Foundry Agent Service Code Interpreter enables the LLM to safely execute Python code for tasks such as creating charts or performing complex data analyses based on user queries. It makes use of natural language processing (NLP), sales data from an SQLite database, and user prompts to automate code generation. The LLM-generated Python code executes within a secure sandbox environment, running on a restricted subset of Python to ensure safe and controlled execution.

Lab Exercise - Python

In this lab, you'll enable the Code Interpreter to execute Python code generated by the LLM.

PythonC#

1. Open the main.py.

2. Define a new instructions file for our agent and add the code intepreter in the agent's toolset. Uncomment the following lines by removing the "# " characters.

   # INSTRUCTIONS_FILE = "instructions/code_interpreter.txt
   
   # code_interpreter = CodeInterpreterTool()
   # toolset.add(code_interpreter)
   

   Warning

   The lines to be uncommented are not adjacent. When removing the # character, ensure you also delete the space that follows it.

3. Review the code in the main.py file.

   After uncommenting, your code should look like this:

   INSTRUCTIONS_FILE = "instructions/function_calling.txt"
   INSTRUCTIONS_FILE = "instructions/file_search.txt"
   INSTRUCTIONS_FILE = "instructions/code_interpreter.txt"
   # INSTRUCTIONS_FILE = "instructions/code_interpreter_multilingual.txt"
   
   
   
   async def add_agent_tools() -> None:
       """Add tools for the agent."""
       font_file_info = None
   
       # Add the functions tool
       toolset.add(functions)
   
       # Add the tents data sheet to a new vector data store
       vector_store = await utilities.create_vector_store(
           agents_client,
           files=[TENTS_DATA_SHEET_FILE],
           vector_store_name="Contoso Product Information Vector Store",
       )
       file_search_tool = FileSearchTool(vector_store_ids=[vector_store.id])
       toolset.add(file_search_tool)
   
       # Add the code interpreter tool
       code_interpreter = CodeInterpreterTool()
       toolset.add(code_interpreter)
   
       # Add multilingual support to the code interpreter
       # font_file_info = await utilities.upload_file(agents_client, utilities.shared_files_path / FONTS_ZIP)
       # code_interpreter.add_file(file_id=font_file_info.id)
   
       return font_file_info
   

1. Open the Program.cs file.

2. Update the creation of the lab to use the Lab2 class.

   await using Lab lab = new Lab3(projectClient, apiDeploymentName);
   

3. Review the Lab2.cs class to see how the Code Interpreter is added to the Tools list.

Review the Instructions

1. Open the shared/instructions/code_interpreter.txt file. This file replaces the instructions used in the previous lab.

2. The Tools section now includes a “Visualization and Code Interpretation” capability, allowing the agent to:

   • Use the code interpreter to run LLM generated Python code. (e.g., for downloading or visualizing data).
   • Create charts and graphs, using the user’s language for labels, titles, and other chart text.
   • Export visualizations as PNG files and data as CSV files.

Run the Agent App

1. Press F5 to run the app.
2. In the terminal, the app will start, and the agent app will prompt you to Enter your query.

Start a Conversation with the Agent

Try these questions:

1. Show sales by region as a pie chart

   Once the task is complete, the file will be saved in the shared/files subfolder. Note that this subfolder is created the first time this task is run, and is never checked into source control.

   Open the folder in VS Code and click on the file to view it. (Tip: in Codespaces, you can Control-Click the link that the agent outputs in its response to view the file.)

   Info

   This might feel like magic, so what’s happening behind the scenes to make it all work?

   Foundry Agent Service orchestrates the following steps:

   1. The LLM generates a SQL query to answer the user's question. In this example, the query is:

      SELECT region, SUM(revenue) AS total_revenue FROM sales_data GROUP BY region;

   2. The LLM asks the agent app to call the async_fetch_sales_data_using_sqlite_query function. The SQL command is executed, and the resulting data is returned to the LLM.

   3. Using the returned data, the LLM writes Python code to create a Pie Chart.
   4. Finally, the Code Interpreter executes the Python code to generate the chart.

2. Download as CSV file

   Once the task is complete, check the shared/files folder to see the downloaded file.

   Info

   The agent inferred from the conversation which file you wanted to create, even though you didn't explicitly specify it.

3. Continue asking questions about Contoso sales data to see the Code Interpreter in action. Few examples:

   • What would be the impact of a shock event (e.g., 20% sales drop in one region) on global sales distribution? Show as a Grouped Bar Chart.
     • Follow up with What if the shock event was 50%?
   • Which regions have sales above or below the average? Show as a Bar Chart with Deviation from Average.
   • Which regions have discounts above or below the average? Show as a Bar Chart with Deviation from Average.
   • Simulate future sales by region using a Monte Carlo simulation to estimate confidence intervals. Show as a Line with Confidence Bands using vivid colors.

Debugging the Code Interpreter

You can’t directly debug the Code Interpreter, but you can gain insight into its behavior by asking the agent to display the code it generates. This helps you understand how it interprets your instructions and can guide you in refining them.

From the terminal, type:

1. show code to see the code generated by the Code Interpreter for the last operation.

Stop the Agent App

When you're done, type exit to clean up the agent resources and stop the app.