Skip to content

API Reference — Probes

probes

Probe factories.

Probes test for the presence of desired behavior. When the evaluator detects the expected behavior, the result is SAFE.

Probes

Factory methods for probe test executions.

behavior staticmethod

Python
behavior(
    *,
    prompt: str,
    evaluator: Evaluator,
    max_turns: int = 25,
    event_handlers: list[ExecutionEventHandler]
    | None = None,
) -> BaseExecution
Python
behavior(
    *,
    prompts: list[str],
    evaluator: Evaluator,
    max_turns: int = 25,
    event_handlers: list[ExecutionEventHandler]
    | None = None,
) -> BaseExecution
Python
behavior(
    *,
    driver: PromptDriver,
    evaluator: Evaluator,
    max_turns: int = 25,
    event_handlers: list[ExecutionEventHandler]
    | None = None,
) -> BaseExecution
Python
behavior(
    *,
    prompt=None,
    prompts=None,
    driver=None,
    evaluator,
    max_turns=25,
    event_handlers=None,
)

Probe whether the agent exhibits desired behavior.

Exactly one of prompt, prompts, or driver must be provided.

Parameters:

Name Type Description Default
prompt str | None

A single prompt string.

 None
prompts list[str] | None

A list of prompt strings.

 None
driver PromptDriver | None

A pre-built prompt driver.

 None
evaluator Evaluator

What behavior to check for.

 required
max_turns int

Maximum prompt-response exchanges before returning ERROR. Defaults to 25.

 25
event_handlers list[ExecutionEventHandler] | None

Optional additional handlers.

 None

Returns:

Name Type Description
BaseExecution BaseExecution

Ready to execute with execute_async(adapter=...).

Raises:

Type Description
ValueError

If more than one or none of prompt, prompts, and driver are provided.
Source code in rampart/probes/__init__.py
Python
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
@staticmethod
def behavior(
    *,
    prompt: str | None = None,
    prompts: list[str] | None = None,
    driver: PromptDriver | None = None,
    evaluator: Evaluator,
    max_turns: int = 25,
    event_handlers: list[ExecutionEventHandler] | None = None,
) -> BaseExecution:
    """Probe whether the agent exhibits desired behavior.

    Exactly one of ``prompt``, ``prompts``, or ``driver`` must be
    provided.

    Args:
        prompt (str | None): A single prompt string.
        prompts (list[str] | None): A list of prompt strings.
        driver (PromptDriver | None): A pre-built prompt driver.
        evaluator (Evaluator): What behavior to check for.
        max_turns (int): Maximum prompt-response exchanges before
            returning ERROR. Defaults to 25.
        event_handlers (list[ExecutionEventHandler] | None): Optional
            additional handlers.

    Returns:
        BaseExecution: Ready to execute with execute_async(adapter=...).

    Raises:
        ValueError: If more than one or none of ``prompt``,
            ``prompts``, and ``driver`` are provided.
    """
    given = sum(x is not None for x in (prompt, prompts, driver))
    if given != 1:
        msg = "Specify exactly one of 'prompt', 'prompts', or 'driver'."
        raise ValueError(
            msg,
        )
    if prompt is not None:
        resolved_driver = coerce_driver(prompt)
    elif prompts is not None:
        resolved_driver = coerce_driver(prompts)
    else:
        assert driver is not None  # noqa: S101  — type narrowing
        resolved_driver = driver
    return SingleTurnExecution(
        driver=resolved_driver,
        evaluator=evaluator,
        max_turns=max_turns,
        event_handlers=event_handlers,
    )

SingleTurnExecution

Python
SingleTurnExecution(
    *, driver, evaluator, max_turns=25, event_handlers=None
)

Bases: BaseExecution

Executes a probe: send prompts, evaluate, resolve as probe.

Inherits BaseExecution. No injection phase — just session creation, prompt driving, evaluation, and cleanup. The lifecycle skeleton (including InfrastructureError handling) is owned by BaseExecution.

Parameters:

Name Type Description Default
driver PromptDriver

How to drive the conversation.

 required
evaluator Evaluator

What behavior to check for.

 required
max_turns int

Maximum prompt-response exchanges before returning ERROR. Defaults to 25.

 25
event_handlers list[ExecutionEventHandler] | None

Additional handlers beyond the framework defaults.

 None
Source code in rampart/probes/_single_turn.py
Python
50
51
52
53
54
55
56
57
58
59
60
61
def __init__(
    self,
    *,
    driver: PromptDriver,
    evaluator: Evaluator,
    max_turns: int = 25,
    event_handlers: list[ExecutionEventHandler] | None = None,
) -> None:
    super().__init__(event_handlers=event_handlers)
    self._driver = driver
    self._evaluator = evaluator
    self._max_turns = max_turns

strategy_name property

Python
strategy_name

Identifies this as a probe execution in results and reports.