Sematic Control Filter

The semantic control filter is a method to filter the controls based on the semantic similarity between the agent's plan and the control's text using their embeddings.

Configuration

To activate the semantic control filtering, you need to add SEMANTIC to the CONTROL_FILTER list in the config_dev.yaml file. Below is the detailed sematic control filter configuration in the config_dev.yaml file:

  • CONTROL_FILTER: A list of filtering methods that you want to apply to the controls. To activate the semantic control filtering, add SEMANTIC to the list.
  • CONTROL_FILTER_TOP_K_SEMANTIC: The number of controls to keep after filtering.
  • CONTROL_FILTER_MODEL_SEMANTIC_NAME: The control filter model name for semantic similarity. By default, it is set to "all-MiniLM-L6-v2".

Reference

Bases: BasicControlFilter

A class that represents a semantic model for control filtering.

control_filter(control_dicts, plans, top_k)

Filters control items based on their similarity to a set of keywords.

Parameters:
  • control_dicts –

    The dictionary of control items to be filtered.

  • plans –

    The list of plans to be used for filtering.

  • top_k –

    The number of top control items to return.
Returns:
  • –

    The filtered control items.
Source code in automator/ui_control/control_filter.py 
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
def control_filter(self, control_dicts, plans, top_k):
    """
    Filters control items based on their similarity to a set of keywords.
    :param control_dicts: The dictionary of control items to be filtered.
    :param plans: The list of plans to be used for filtering.
    :param top_k: The number of top control items to return.
    :return: The filtered control items.
    """
    scores_items = []
    filtered_control_dict = {}

    for label, control_item in control_dicts.items():
        control_text = control_item.element_info.name.lower()
        score = self.control_filter_score(control_text, plans)
        scores_items.append((label, score))
    topk_scores_items = heapq.nlargest(top_k, (scores_items), key=lambda x: x[1])
    topk_items = [
        (score_item[0], score_item[1]) for score_item in topk_scores_items
    ]

    for label, control_item in control_dicts.items():
        if label in topk_items:
            filtered_control_dict[label] = control_item
    return filtered_control_dict

control_filter_score(control_text, plans)

Calculates the score for a control item based on the similarity between its text and a set of keywords.

Parameters:
  • control_text –

    The text of the control item.

  • plans –

    The plan to be used for calculating the similarity.
Returns:
  • –

    The score (0-1) indicating the similarity between the control text and the keywords.
Source code in automator/ui_control/control_filter.py 
197
198
199
200
201
202
203
204
205
206
207
def control_filter_score(self, control_text, plans):
    """
    Calculates the score for a control item based on the similarity between its text and a set of keywords.
    :param control_text: The text of the control item.
    :param plans: The plan to be used for calculating the similarity.
    :return: The score (0-1) indicating the similarity between the control text and the keywords.
    """

    plan_embedding = self.get_embedding(plans)
    control_text_embedding = self.get_embedding(control_text)
    return max(self.cos_sim(control_text_embedding, plan_embedding).tolist()[0])