Lab 2 Intro to the Code Interpreter

Introduction

The Azure AI Agent Service Code Interpreter enables the LLM to safely execute Python code for tasks such as creating charts or performing complex data analyses based on user queries. It makes use of natural language processing (NLP), sales data from an SQLite database, and user prompts to automate code generation. The LLM-generated Python code executes within a secure sandbox environment, running on a restricted subset of Python to ensure safe and controlled execution.

Lab Exercise

In this lab, you'll enable the Code Interpreter to execute Python code generated by the LLM.

1. Open the main.py.

2. Define a new instructions file for our agent: uncomment the following lines by removing the "# " characters

   # INSTRUCTIONS_FILE = "instructions/instructions_code_interpreter.txt
   
   # code_interpreter = CodeInterpreterTool()
   # toolset.add(code_interpreter)
   

   Warning

   The lines to be uncommented are not adjacent. When removing the # character, ensure you also delete the space that follows it.

3. Review the code in the main.py file.

   After uncommenting, your code should look like this:

   INSTRUCTIONS_FILE = "instructions/instructions_function_calling.txt"
   INSTRUCTIONS_FILE = "instructions/instructions_code_interpreter.txt"
   # INSTRUCTIONS_FILE = "instructions/instructions_file_search.txt"
   # INSTRUCTIONS_FILE = "instructions/instructions_bing_grounding.txt"
   
   
   async def add_agent_tools():
       """Add tools for the agent."""
   
       # Add the functions tool
       toolset.add(functions)
   
       # Add the code interpreter tool
       code_interpreter = CodeInterpreterTool()
       toolset.add(code_interpreter)
   
       # Add the tents data sheet to a new vector data store
       # vector_store = await utilities.create_vector_store(
       #     project_client,
       #     files=[TENTS_DATA_SHEET_FILE],
       #     vector_name_name="Contoso Product Information Vector Store",
       # )
       # file_search_tool = FileSearchTool(vector_store_ids=[vector_store.id])
       # toolset.add(file_search_tool)
   
       # Add the Bing grounding tool
       # bing_connection = await project_client.connections.get(connection_name=BING_CONNECTION_NAME)
       # bing_grounding = BingGroundingTool(connection_id=bing_connection.id)
       # toolset.add(bing_grounding)
   

Review the Instructions

1. Open the src/workshop/instructions/instructions_code_interpreter.txt file. This file replaces the instructions used in the previous lab.

2. The Tools section now includes a “Visualization and Code Interpretation” capability, allowing the agent to:

   • Use the code interpreter to run programs generated by the LLM (e.g., for downloading or visualizing data).
   • Create charts and graphs, using the user’s language for labels, titles, and other chart text.
   • Export visualizations as PNG files and data as CSV files.

Run the Agent App

1. Press F5 to run the app.
2. In the terminal, the app will start, and the agent app will prompt you to Enter your query.

Start a Conversation with the Agent

Try these questions:

1. Show sales by region as a pie chart

   Once the task is complete, the pie chart image will be saved in the files subfolder. Note that this subfolder is created the first time this task is run, and is never checked into source control.

   Open the folder in VS Code and click on the image file to view it. (Tip: in Codespaces, you can Control-Click the link that the agent outputs in its response to view the file.)

   Info

   This might feel like magic, so what’s happening behind the scenes to make it all work?

   Azure AI Agent Service orchestrates the following steps:

   1. The LLM generates a SQL query to answer the user's question. In this example, the query is:

      SELECT region, SUM(revenue) AS total_revenue FROM sales_data GROUP BY region;

   2. The LLM asks the agent app to call the async_fetch_sales_data_using_sqlite_query function. The SQL command is executed, and the resulting data is returned to the LLM.

   3. Using the returned data, the LLM writes Python code to create a Pie Chart.
   4. Finally, the Code Interpreter executes the Python code to generate the chart.

2. Download the sales by region data

   Once the task is complete, check the files folder to see the downloaded file.

   Info

   By default, the instructions specify that data downloads in CSV format. You can request other formats, such as JSON or Excel, by including the desired format in your query (e.g., ‘Download as JSON’).

3. Download as JSON

   Once the task is complete, check the files folder to see the downloaded file.

   Info

   The agent inferred from the conversation which file you wanted to create, even though you didn't explicitly specify it.

4. Continue asking questions about Contoso sales data to see the Code Interpreter in action.

Stop the Agent App

When you're done, type exit to clean up the agent resources and stop the app.