{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Human-in-the-Loop\n",
    "\n",
    "In the previous section [Teams](./teams.ipynb), we have seen how to create, observe,\n",
    "and control a team of agents.\n",
    "This section will focus on how to interact with the team from your application,\n",
    "and provide human feedback to the team.\n",
    "\n",
    "There are two main ways to interact with the team from your application:\n",
    "\n",
    "1. During a team's run -- execution of {py:meth}`~autogen_agentchat.teams.BaseGroupChat.run` or {py:meth}`~autogen_agentchat.teams.BaseGroupChat.run_stream`, provide feedback through a {py:class}`~autogen_agentchat.agents.UserProxyAgent`.\n",
    "2. Once the run terminates, provide feedback through input to the next call to {py:meth}`~autogen_agentchat.teams.BaseGroupChat.run` or {py:meth}`~autogen_agentchat.teams.BaseGroupChat.run_stream`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Providing Feedback During a Run\n",
    "\n",
    "The {py:class}`~autogen_agentchat.agents.UserProxyAgent` is a special built-in agent\n",
    "that acts as a proxy for a user to provide feedback to the team.\n",
    "\n",
    "To use the {py:class}`~autogen_agentchat.agents.UserProxyAgent`, you can create an instance of it\n",
    "and include it in the team before running the team.\n",
    "The team will decide when to call the {py:class}`~autogen_agentchat.agents.UserProxyAgent`\n",
    "to ask for feedback from the user.\n",
    "\n",
    "The following diagram illustrates how you can use \n",
    "{py:class}`~autogen_agentchat.agents.UserProxyAgent`\n",
    "to get feedback from the user during a team's run:\n",
    "\n",
    "![human-in-the-loop-user-proxy](./human-in-the-loop-user-proxy.svg)\n",
    "\n",
    "The bold arrows indicates the flow of control during a team's run:\n",
    "when the team calls the {py:class}`~autogen_agentchat.agents.UserProxyAgent`,\n",
    "it transfers the control to the application/user, and waits for the feedback;\n",
    "once the feedback is provided, the control is transferred back to the team\n",
    "and the team continues its execution.\n",
    "\n",
    "```{note}\n",
    "When {py:class}`~autogen_agentchat.agents.UserProxyAgent` is called during a run,\n",
    "it blocks the execution of the team until the user provides feedback or errors out.\n",
    "This will hold up the team's progress and put the team in an unstable state\n",
    "that cannot be saved or resumed.\n",
    "```\n",
    "\n",
    "Due to the blocking nature of this approach, it is recommended to use it only for short interactions\n",
    "that require immediate feedback from the user, such as asking for approval or disapproval\n",
    "with a button click, or an alert requiring immediate attention otherwise failing the task.\n",
    "\n",
    "Here is an example of how to use the {py:class}`~autogen_agentchat.agents.UserProxyAgent`\n",
    "in a {py:class}`~autogen_agentchat.teams.RoundRobinGroupChat` for a poetry generation task:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------- user ----------\n",
      "Write a 4-line poem about the ocean.\n",
      "---------- assistant ----------\n",
      "Waves whisper secrets to the shore’s embrace,  \n",
      "A dance of blue under the sun's warm grace.  \n",
      "Endless horizons where dreams take flight,  \n",
      "The ocean's heart glimmers, a canvas of light.  \n",
      "TERMINATE\n",
      "[Prompt tokens: 46, Completion tokens: 49]\n",
      "---------- user_proxy ----------\n",
      "APPROVE\n",
      "---------- Summary ----------\n",
      "Number of messages: 3\n",
      "Finish reason: Text 'APPROVE' mentioned\n",
      "Total prompt tokens: 46\n",
      "Total completion tokens: 49\n",
      "Duration: 6.64 seconds\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a 4-line poem about the ocean.', type='TextMessage'), TextMessage(source='assistant', models_usage=RequestUsage(prompt_tokens=46, completion_tokens=49), content=\"Waves whisper secrets to the shore’s embrace,  \\nA dance of blue under the sun's warm grace.  \\nEndless horizons where dreams take flight,  \\nThe ocean's heart glimmers, a canvas of light.  \\nTERMINATE\", type='TextMessage'), TextMessage(source='user_proxy', models_usage=None, content='APPROVE', type='TextMessage')], stop_reason=\"Text 'APPROVE' mentioned\")"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from autogen_agentchat.agents import AssistantAgent, UserProxyAgent\n",
    "from autogen_agentchat.conditions import TextMentionTermination\n",
    "from autogen_agentchat.teams import RoundRobinGroupChat\n",
    "from autogen_agentchat.ui import Console\n",
    "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
    "\n",
    "# Create the agents.\n",
    "model_client = OpenAIChatCompletionClient(model=\"gpt-4o-mini\")\n",
    "assistant = AssistantAgent(\"assistant\", model_client=model_client)\n",
    "user_proxy = UserProxyAgent(\"user_proxy\", input_func=input)  # Use input() to get user input from console.\n",
    "\n",
    "# Create the termination condition which will end the conversation when the user says \"APPROVE\".\n",
    "termination = TextMentionTermination(\"APPROVE\")\n",
    "\n",
    "# Create the team.\n",
    "team = RoundRobinGroupChat([assistant, user_proxy], termination_condition=termination)\n",
    "\n",
    "# Run the conversation and stream to the console.\n",
    "stream = team.run_stream(task=\"Write a 4-line poem about the ocean.\")\n",
    "# Use asyncio.run(...) when running in a script.\n",
    "await Console(stream)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the console output, you can see the team solicited feedback from the user\n",
    "through `user_proxy` to approve the generated poem.\n",
    "\n",
    "You can provide your own input function to the {py:class}`~autogen_agentchat.agents.UserProxyAgent`\n",
    "to customize the feedback process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Providing Feedback to the Next Run\n",
    "\n",
    "Often times, an application or a user interacts with the team of agents in an interactive loop:\n",
    "the team runs until termination, \n",
    "the application or user provides feedback, and the team runs again with the feedback.\n",
    "\n",
    "This approach is useful in a persisted session\n",
    "with asynchronous communication between the team and the application/user:\n",
    "Once a team finishes a run, the application saves the state of the team,\n",
    "puts it in a persistent storage, and resumes the team when the feedback arrives.\n",
    "\n",
    "```{note}\n",
    "For how to save and load the state of a team, please refer to [Managing State](./state.ipynb).\n",
    "This section will focus on the feedback mechanisms.\n",
    "```\n",
    "\n",
    "The following diagram illustrates the flow of control in this approach:\n",
    "\n",
    "![human-in-the-loop-termination](./human-in-the-loop-termination.svg)\n",
    "\n",
    "There are two ways to implement this approach:\n",
    "\n",
    "- Set the maximum number of turns so that the team always stops after the specified number of turns.\n",
    "- Use termination conditions such as {py:class}`~autogen_agentchat.conditions.TextMentionTermination` and {py:class}`~autogen_agentchat.conditions.HandoffTermination` to allow the team to decide when to stop and give control back, given the team's internal state.\n",
    "\n",
    "You can use both methods together to achieve your desired behavior."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Max Turns\n",
    "\n",
    "This method allows you to pause the team for user input by setting a maximum number of turns. For instance, you can configure the team to stop after the first agent responds by setting `max_turns` to 1. This is particularly useful in scenarios where continuous user engagement is required, such as in a chatbot.\n",
    "\n",
    "To implement this, set the `max_turns` parameter in the {py:meth}`~autogen_agentchat.teams.RoundRobinGroupChat` constructor.\n",
    "\n",
    "```python\n",
    "team = RoundRobinGroupChat([...], max_turns=1)\n",
    "```\n",
    "\n",
    "Once the team stops, the turn count will be reset. When you resume the team,\n",
    "it will start from 0 again. However, the team's internal state will be preserved,\n",
    "for example, the {py:class}`~autogen_agentchat.teams.RoundRobinGroupChat` will\n",
    "resume from the next agent in the list with the same conversation history.\n",
    "\n",
    "```{note}\n",
    "`max_turn` is specific to the team class and is currently only supported by\n",
    "{py:class}`~autogen_agentchat.teams.RoundRobinGroupChat`, {py:class}`~autogen_agentchat.teams.SelectorGroupChat`, and {py:class}`~autogen_agentchat.teams.Swarm`.\n",
    "When used with termination conditions, the team will stop when either condition is met.\n",
    "```\n",
    "\n",
    "Here is an example of how to use `max_turns` in a {py:class}`~autogen_agentchat.teams.RoundRobinGroupChat` for a poetry generation task\n",
    "with a maximum of 1 turn:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------- user ----------\n",
      "Write a 4-line poem about the ocean.\n",
      "---------- assistant ----------\n",
      "Endless waves in a dance with the shore,  \n",
      "Whispers of secrets in tales from the roar,  \n",
      "Beneath the vast sky, where horizons blend,  \n",
      "The ocean’s embrace is a timeless friend.  \n",
      "TERMINATE\n",
      "[Prompt tokens: 46, Completion tokens: 48]\n",
      "---------- Summary ----------\n",
      "Number of messages: 2\n",
      "Finish reason: Maximum number of turns 1 reached.\n",
      "Total prompt tokens: 46\n",
      "Total completion tokens: 48\n",
      "Duration: 1.63 seconds\n",
      "---------- user ----------\n",
      "Can you make it about a person and its relationship with the ocean\n",
      "---------- assistant ----------\n",
      "She walks along the tide, where dreams intertwine,  \n",
      "With every crashing wave, her heart feels aligned,  \n",
      "In the ocean's embrace, her worries dissolve,  \n",
      "A symphony of solace, where her spirit evolves.  \n",
      "TERMINATE\n",
      "[Prompt tokens: 117, Completion tokens: 49]\n",
      "---------- Summary ----------\n",
      "Number of messages: 2\n",
      "Finish reason: Maximum number of turns 1 reached.\n",
      "Total prompt tokens: 117\n",
      "Total completion tokens: 49\n",
      "Duration: 1.21 seconds\n"
     ]
    }
   ],
   "source": [
    "from autogen_agentchat.agents import AssistantAgent\n",
    "from autogen_agentchat.teams import RoundRobinGroupChat\n",
    "from autogen_agentchat.ui import Console\n",
    "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
    "\n",
    "# Create the agents.\n",
    "model_client = OpenAIChatCompletionClient(model=\"gpt-4o-mini\")\n",
    "assistant = AssistantAgent(\"assistant\", model_client=model_client)\n",
    "\n",
    "# Create the team setting a maximum number of turns to 1.\n",
    "team = RoundRobinGroupChat([assistant], max_turns=1)\n",
    "\n",
    "task = \"Write a 4-line poem about the ocean.\"\n",
    "while True:\n",
    "    # Run the conversation and stream to the console.\n",
    "    stream = team.run_stream(task=task)\n",
    "    # Use asyncio.run(...) when running in a script.\n",
    "    await Console(stream)\n",
    "    # Get the user response.\n",
    "    task = input(\"Enter your feedback (type 'exit' to leave): \")\n",
    "    if task.lower().strip() == \"exit\":\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the team stopped immediately after one agent responded."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Termination Conditions\n",
    "\n",
    "We have already seen several examples of termination conditions in the previous sections.\n",
    "In this section, we focus on {py:class}`~autogen_agentchat.conditions.HandoffTermination`\n",
    "which stops the team when an agent sends a {py:class}`~autogen_agentchat.messages.HandoffMessage` message.\n",
    "\n",
    "Let's create a team with a single {py:class}`~autogen_agentchat.agents.AssistantAgent` agent\n",
    "with a handoff setting, and run the team with a task that requires additional input from the user\n",
    "because the agent doesn't have relevant tools to continue processing the task.\n",
    "\n",
    "```{note}\n",
    "The model used with {py:class}`~autogen_agentchat.agents.AssistantAgent` must support tool call\n",
    "to use the handoff feature.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------- user ----------\n",
      "What is the weather in New York?\n",
      "---------- lazy_assistant ----------\n",
      "[FunctionCall(id='call_nSjgvWCUYo5ccacBz7yzrPLN', arguments='{}', name='transfer_to_user')]\n",
      "[Prompt tokens: 68, Completion tokens: 12]\n",
      "---------- lazy_assistant ----------\n",
      "[FunctionExecutionResult(content='Transfer to user.', call_id='call_nSjgvWCUYo5ccacBz7yzrPLN')]\n",
      "---------- lazy_assistant ----------\n",
      "Transfer to user.\n",
      "---------- Summary ----------\n",
      "Number of messages: 4\n",
      "Finish reason: Handoff to user from lazy_assistant detected.\n",
      "Total prompt tokens: 68\n",
      "Total completion tokens: 12\n",
      "Duration: 0.75 seconds\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TaskResult(messages=[TextMessage(source='user', models_usage=None, content='What is the weather in New York?', type='TextMessage'), ToolCallRequestEvent(source='lazy_assistant', models_usage=RequestUsage(prompt_tokens=68, completion_tokens=12), content=[FunctionCall(id='call_nSjgvWCUYo5ccacBz7yzrPLN', arguments='{}', name='transfer_to_user')], type='ToolCallRequestEvent'), ToolCallExecutionEvent(source='lazy_assistant', models_usage=None, content=[FunctionExecutionResult(content='Transfer to user.', call_id='call_nSjgvWCUYo5ccacBz7yzrPLN')], type='ToolCallExecutionEvent'), HandoffMessage(source='lazy_assistant', models_usage=None, target='user', content='Transfer to user.', type='HandoffMessage')], stop_reason='Handoff to user from lazy_assistant detected.')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from autogen_agentchat.agents import AssistantAgent\n",
    "from autogen_agentchat.base import Handoff\n",
    "from autogen_agentchat.conditions import HandoffTermination, TextMentionTermination\n",
    "from autogen_agentchat.teams import RoundRobinGroupChat\n",
    "from autogen_agentchat.ui import Console\n",
    "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
    "\n",
    "# Create an OpenAI model client.\n",
    "model_client = OpenAIChatCompletionClient(\n",
    "    model=\"gpt-4o-2024-08-06\",\n",
    "    # api_key=\"sk-...\", # Optional if you have an OPENAI_API_KEY env variable set.\n",
    ")\n",
    "\n",
    "# Create a lazy assistant agent that always hands off to the user.\n",
    "lazy_agent = AssistantAgent(\n",
    "    \"lazy_assistant\",\n",
    "    model_client=model_client,\n",
    "    handoffs=[Handoff(target=\"user\", message=\"Transfer to user.\")],\n",
    "    system_message=\"Always transfer to user when you don't know the answer. Respond 'TERMINATE' when task is complete.\",\n",
    ")\n",
    "\n",
    "# Define a termination condition that checks for handoff message targetting helper and text \"TERMINATE\".\n",
    "handoff_termination = HandoffTermination(target=\"user\")\n",
    "text_termination = TextMentionTermination(\"TERMINATE\")\n",
    "combined_termination = handoff_termination | text_termination\n",
    "\n",
    "# Create a single-agent team.\n",
    "lazy_agent_team = RoundRobinGroupChat([lazy_agent], termination_condition=combined_termination)\n",
    "\n",
    "# Run the team and stream to the console.\n",
    "task = \"What is the weather in New York?\"\n",
    "await Console(lazy_agent_team.run_stream(task=task))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the team stopped due to the handoff message was detected.\n",
    "Let's continue the team by providing the information the agent needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------- user ----------\n",
      "The weather in New York is sunny.\n",
      "---------- lazy_assistant ----------\n",
      "Great to hear that it's sunny in New York! Is there anything else you'd like to know or discuss?\n",
      "[Prompt tokens: 109, Completion tokens: 23]\n",
      "---------- lazy_assistant ----------\n",
      "TERMINATE\n",
      "[Prompt tokens: 138, Completion tokens: 5]\n",
      "---------- Summary ----------\n",
      "Number of messages: 3\n",
      "Finish reason: Text 'TERMINATE' mentioned\n",
      "Total prompt tokens: 247\n",
      "Total completion tokens: 28\n",
      "Duration: 1.44 seconds\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TaskResult(messages=[TextMessage(source='user', models_usage=None, content='The weather in New York is sunny.', type='TextMessage'), TextMessage(source='lazy_assistant', models_usage=RequestUsage(prompt_tokens=109, completion_tokens=23), content=\"Great to hear that it's sunny in New York! Is there anything else you'd like to know or discuss?\", type='TextMessage'), TextMessage(source='lazy_assistant', models_usage=RequestUsage(prompt_tokens=138, completion_tokens=5), content='TERMINATE', type='TextMessage')], stop_reason=\"Text 'TERMINATE' mentioned\")"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await Console(lazy_agent_team.run_stream(task=\"The weather in New York is sunny.\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the team continued after the user provided the information."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}