Grader: skill-invocation

Taxonomy

Property	Value
Determinism	static
Cost	free
Reference	reference-free
Temporal scope	trajectory-level
Score kind	code

Config

graders:
  - type: skill-invocation
    config:
      required:
        - code-review
        - summarization
      disallowed:
        - web-search

Field	Type	Required	Default	Description
required	string[]	No	—	Skills that must be activated. Fails if any are missing
disallowed	string[]	No	—	Skills that must NOT be activated. Fails if any are found

At least one of required or disallowed must be provided. A skill cannot appear in both lists — the grader throws an error if any overlap is detected.

Behavior

Scans trajectory events for skill_activation events and collects the activated skill names.

• Skill names are matched with exact string comparison (no regex).
• A required entry is satisfied when at least one activation matches it.
• A disallowed entry is violated when any activation matches it.

Passes when all required skills were activated and no disallowed skills were activated. Fails otherwise.

Use cases

# Agent must invoke the code-review skill
- type: skill-invocation
  config:
    required:
      - code-review

# Agent must not use the web-search skill
- type: skill-invocation
  config:
    disallowed:
      - web-search

# Combine required and disallowed (no overlaps allowed)
- type: skill-invocation
  config:
    required:
      - summarization
    disallowed:
      - web-search