Agent Prompter

The Prompter is a key component of the UFO framework, responsible for constructing prompts for the LLM to generate responses. The Prompter is implemented in the ufo/prompts folder. Each agent has its own Prompter class that defines the structure of the prompt and the information to be fed to the LLM.

Components

A prompt fed to the LLM usually a list of dictionaries, where each dictionary contains the following keys:

Key Description
role The role of the text in the prompt, can be system, user, or assistant.
content The content of the text for the specific role.

Tip

You may find the official documentation helpful for constructing the prompt.

In the __init__ method of the Prompter class, you can define the template of the prompt for each component, and the final prompt message is constructed by combining the templates of each component using the prompt_construction method.

System Prompt

The system prompt use the template configured in the config_dev.yaml file for each agent. It usually contains the instructions for the agent's role, action, tips, reponse format, etc. You need use the system_prompt_construction method to construct the system prompt.

Prompts on the API instructions, and demonstration examples are also included in the system prompt, which are constructed by the api_prompt_helper and examples_prompt_helper methods respectively. Below is the sub-components of the system prompt:

Component Description Method
apis The API instructions for the agent. api_prompt_helper
examples The demonstration examples for the agent. examples_prompt_helper

User Prompt

The user prompt is constructed based on the information from the agent's observation, external knowledge, and Blackboard. You can use the user_prompt_construction method to construct the user prompt. Below is the sub-components of the user prompt:

Component Description Method
observation The observation of the agent. user_content_construction
retrieved_docs The knowledge retrieved from the external knowledge base. retrived_documents_prompt_helper
blackboard The information stored in the Blackboard. blackboard_to_prompt

Reference

You can find the implementation of the Prompter in the ufo/prompts folder. Below is the basic structure of the Prompter class:

Bases: ABC

The BasicPrompter class is the abstract class for the prompter.

Initialize the BasicPrompter.

Parameters:
  • is_visual (bool) –

    Whether the request is for visual model.

  • prompt_template (str) –

    The path of the prompt template.

  • example_prompt_template (str) –

    The path of the example prompt template.
Source code in prompter/basic.py 
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
def __init__(
    self, is_visual: bool, prompt_template: str, example_prompt_template: str
):
    """
    Initialize the BasicPrompter.
    :param is_visual: Whether the request is for visual model.
    :param prompt_template: The path of the prompt template.
    :param example_prompt_template: The path of the example prompt template.
    """
    self.is_visual = is_visual
    if prompt_template:
        self.prompt_template = self.load_prompt_template(prompt_template, is_visual)
    else:
        self.prompt_template = ""
    if example_prompt_template:
        self.example_prompt_template = self.load_prompt_template(
            example_prompt_template, is_visual
        )
    else:
        self.example_prompt_template = ""

api_prompt_helper()

A helper function to construct the API list and descriptions for the prompt.

Source code in prompter/basic.py 
139
140
141
142
143
144
def api_prompt_helper(self) -> str:
    """
    A helper function to construct the API list and descriptions for the prompt.
    """

    pass

examples_prompt_helper()

A helper function to construct the examples prompt for in-context learning.

Source code in prompter/basic.py 
132
133
134
135
136
137
def examples_prompt_helper(self) -> str:
    """
    A helper function to construct the examples prompt for in-context learning.
    """

    pass

load_prompt_template(template_path, is_visual=None) staticmethod

Load the prompt template.

Returns:
  • Dict[str, str] –

    The prompt template.
Source code in prompter/basic.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
@staticmethod
def load_prompt_template(template_path: str, is_visual=None) -> Dict[str, str]:
    """
    Load the prompt template.
    :return: The prompt template.
    """

    if is_visual == None:
        path = template_path
    else:
        path = template_path.format(
            mode="visual" if is_visual == True else "nonvisual"
        )

    if not path:
        return {}

    if os.path.exists(path):
        try:
            prompt = yaml.safe_load(open(path, "r", encoding="utf-8"))
        except yaml.YAMLError as exc:
            print_with_color(f"Error loading prompt template: {exc}", "yellow")
    else:
        raise FileNotFoundError(f"Prompt template not found at {path}")

    return prompt

prompt_construction(system_prompt, user_content) staticmethod

Construct the prompt for summarizing the experience into an example.

Parameters:
  • user_content (List[Dict[str, str]]) –

    The user content. return: The prompt for summarizing the experience into an example.
Source code in prompter/basic.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
@staticmethod
def prompt_construction(
    system_prompt: str, user_content: List[Dict[str, str]]
) -> List:
    """
    Construct the prompt for summarizing the experience into an example.
    :param user_content: The user content.
    return: The prompt for summarizing the experience into an example.
    """

    system_message = {"role": "system", "content": system_prompt}

    user_message = {"role": "user", "content": user_content}

    prompt_message = [system_message, user_message]

    return prompt_message

retrived_documents_prompt_helper(header, separator, documents) staticmethod

Construct the prompt for retrieved documents.

Parameters:
  • header (str) –

    The header of the prompt.

  • separator (str) –

    The separator of the prompt.

  • documents (List[str]) –

    The retrieved documents. return: The prompt for retrieved documents.
Source code in prompter/basic.py 
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
@staticmethod
def retrived_documents_prompt_helper(
    header: str, separator: str, documents: List[str]
) -> str:
    """
    Construct the prompt for retrieved documents.
    :param header: The header of the prompt.
    :param separator: The separator of the prompt.
    :param documents: The retrieved documents.
    return: The prompt for retrieved documents.
    """

    if header:
        prompt = "\n<{header}:>\n".format(header=header)
    else:
        prompt = ""
    for i, document in enumerate(documents):
        if separator:
            prompt += "[{separator} {i}:]".format(separator=separator, i=i + 1)
            prompt += "\n"
        prompt += document
        prompt += "\n\n"
    return prompt

system_prompt_construction() abstractmethod

Construct the system prompt for LLM.

Source code in prompter/basic.py 
108
109
110
111
112
113
114
@abstractmethod
def system_prompt_construction(self) -> str:
    """
    Construct the system prompt for LLM.
    """

    pass

user_content_construction() abstractmethod

Construct the full user content for LLM, including the user prompt and images.

Source code in prompter/basic.py 
124
125
126
127
128
129
130
@abstractmethod
def user_content_construction(self) -> str:
    """
    Construct the full user content for LLM, including the user prompt and images.
    """

    pass

user_prompt_construction() abstractmethod

Construct the textual user prompt for LLM based on the user field in the prompt template.

Source code in prompter/basic.py 
116
117
118
119
120
121
122
@abstractmethod
def user_prompt_construction(self) -> str:
    """
    Construct the textual user prompt for LLM based on the `user` field in the prompt template.
    """

    pass

Tip

You can customize the Prompter class to tailor the prompt to your requirements.