Follower Mode
Follower mode enables UFO to execute a predefined list of steps in natural language. Unlike normal mode where the agent generates its own plan, follower mode creates an AppAgent that follows user-provided steps to interact with applications. This mode is particularly useful for debugging, software testing, and verification.
Quick Start
Step 1: Create a Plan File
Create a JSON plan file containing the steps for the agent to follow:
| Field | Description | Type |
|---|---|---|
| task | The task description. | String |
| steps | The list of steps for the agent to follow. | List of Strings |
| object | The application or file to interact with. | String |
Example plan file:
{
"task": "Type in a text of 'Test For Fun' with heading 1 level",
"steps":
[
"1.type in 'Test For Fun'",
"2.Select the 'Test For Fun' text",
"3.Click 'Home' tab to show the 'Styles' ribbon tab",
"4.Click 'Styles' ribbon tab to show the style 'Heading 1'",
"5.Click 'Heading 1' style to apply the style to the selected text"
],
"object": "draft.docx"
}
The object field specifies the application or file the agent will interact with. This object should be opened and accessible before starting follower mode.
Step 2: Start Follower Mode
Run the following command:
# Assume you are in the cloned UFO folder
python -m ufo --task {task_name} --mode follower --plan {plan_file}
Parameters:
- {task_name}: Name for this task execution (used for logging)
- {plan_file}: Path to the plan JSON file
Step 3: Run in Batch (Optional)
To execute multiple plan files sequentially, provide a folder containing multiple plan files:
# Assume you are in the cloned UFO folder
python -m ufo --task {task_name} --mode follower --plan {plan_folder}
UFO will automatically detect and execute all plan files in the folder sequentially.
Parameters:
- {task_name}: Name for this batch execution (used for logging)
- {plan_folder}: Path to the folder containing plan JSON files
Evaluation
UFO can automatically evaluate task completion. To enable evaluation, ensure EVA_SESSION is set to True in config/ufo/system.yaml.
Check the evaluation results in logs/{task_name}/evaluation.log.
References
Follower mode uses a PlanReader to parse the plan file and creates a FollowerSession to execute the steps.
PlanReader
The PlanReader is located at ufo/module/sessions/plan_reader.py.
The reader for a plan file.
Initialize a plan reader.
| Parameters: |
|
|---|
Source code in module/sessions/plan_reader.py
18 19 20 21 22 23 24 25 26 27 28 | |
get_close()
Check if the plan is closed.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
30 31 32 33 34 35 36 | |
get_host_agent_request()
Get the request for the host agent.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
75 76 77 78 79 80 81 82 83 84 85 86 87 88 | |
get_host_request()
Get the request for the host agent.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | |
get_initial_request()
Get the initial request in the plan.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
62 63 64 65 66 67 68 69 70 71 72 73 | |
get_operation_object()
Get the operation object in the step.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
54 55 56 57 58 59 60 | |
get_root_path()
Get the root path of the plan.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
148 149 150 151 152 153 154 | |
get_steps()
Get the steps in the plan.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
46 47 48 49 50 51 52 | |
get_support_apps()
Get the support apps in the plan.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
103 104 105 106 107 108 109 | |
get_task()
Get the task name.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
38 39 40 41 42 43 44 | |
next_step()
Get the next step in the plan.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
128 129 130 131 132 133 134 135 136 137 138 | |
task_finished()
Check if the task is finished.
| Returns: |
|
|---|
Source code in module/sessions/plan_reader.py
140 141 142 143 144 145 146 | |
FollowerSession
The FollowerSession is located at ufo/module/sessions/session.py.
Bases: WindowsBaseSession
A session for following a list of plan for action taken. This session is used for the follower agent, which accepts a plan file to follow using the PlanReader.
Initialize a session.
| Parameters: |
|
|---|
Source code in module/sessions/session.py
166 167 168 169 170 171 172 173 174 175 176 177 178 179 | |
create_new_round()
Create a new round.
Source code in module/sessions/session.py
192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 | |
next_request()
Get the request for the new round.
Source code in module/sessions/session.py
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 | |
request_to_evaluate()
Get the request to evaluate. return: The request(s) to evaluate.
Source code in module/sessions/session.py
248 249 250 251 252 253 254 | |