Foundry Scenarios - PyRIT Documentation

The Foundry scenario family provides the RedTeamAgent — a comprehensive red teaming scenario that combines converter-based attacks (encoding/obfuscation), multi-turn attacks (Crescendo, RedTeaming), and strategy composition. It’s organized into difficulty levels: EASY, MODERATE, and DIFFICULT.

For full programming details, see Scenario Parameters.

from pathlib import Path

from pyrit.registry import TargetRegistry
from pyrit.scenario import DatasetConfiguration
from pyrit.scenario.printer.console_printer import ConsoleScenarioResultPrinter
from pyrit.scenario.scenarios.foundry import FoundryStrategy, RedTeamAgent
from pyrit.setup import initialize_from_config_async

await initialize_from_config_async(config_path=Path("pyrit_conf.yaml"))  # type: ignore

objective_target = TargetRegistry.get_registry_singleton().get_instance_by_name("openai_chat")
printer = ConsoleScenarioResultPrinter()

Found default environment files: ['./.pyrit/.env', './.pyrit/.env.local']
Loaded environment file: ./.pyrit/.env
Loaded environment file: ./.pyrit/.env.local

RedTeamAgent¶

Tests a target using a wide range of attack strategies — from simple encoding converters to complex multi-turn conversations. The default dataset is HarmBench.

CLI example:

pyrit_scan foundry.red_team_agent --target openai_chat --strategies base64 --max-dataset-size 1

Available strategies by difficulty:

Difficulty	Strategies
EASY	AnsiAttack, AsciiArt, AsciiSmuggler, Atbash, Base64, Binary, Caesar, CharacterSpace, CharSwap, Diacritic, Flip, Jailbreak, Leetspeak, Morse, ROT13, StringJoin, SuffixAppend, UnicodeConfusable, UnicodeSubstitution, Url
MODERATE	Tense
DIFFICULT	Crescendo, MultiTurn, Pair, Tap
Aggregates	ALL, EASY, MODERATE, DIFFICULT

dataset_config = DatasetConfiguration(dataset_names=["harmbench"], max_dataset_size=1)

scenario = RedTeamAgent()
await scenario.initialize_async(  # type: ignore
    objective_target=objective_target,
    scenario_strategies=[FoundryStrategy.Base64],
    dataset_config=dataset_config,
)

print(f"Scenario: {scenario.name}")
print(f"Atomic attacks: {scenario.atomic_attack_count}")

scenario_result = await scenario.run_async()  # type: ignore

Scenario: RedTeamAgent
Atomic attacks: 2

await printer.print_summary_async(scenario_result)  # type: ignore


====================================================================================================
                                  📊 SCENARIO RESULTS: RedTeamAgent                                  
====================================================================================================

▼ Scenario Information
────────────────────────────────────────────────────────────────────────────────────────────────────
  📋 Scenario Details
    • Name: RedTeamAgent
    • Scenario Version: 1
    • PyRIT Version: 0.12.1.dev0
    • Description:
        RedTeamAgent is a preconfigured scenario that automatically generates multiple AtomicAttack instances based on
        the specified attack strategies. It supports both single-turn attacks (with various converters) and multi-turn
        attacks (Crescendo, RedTeaming), making it easy to quickly test a target against multiple attack vectors. The
        scenario can expand difficulty levels (EASY, MODERATE, DIFFICULT) into their constituent attack strategies, or
        you can specify individual strategies directly. This scenario is designed for use with the Foundry AI Red
        Teaming Agent library, providing a consistent PyRIT contract for their integration.

  🎯 Target Information
    • Target Type: OpenAIChatTarget
    • Target Model: gpt-4o
    • Target Endpoint: https://pyrit-dev.openai.azure.com/openai/v1

  📊 Scorer Information
    ▸ Scorer Identifier
      • Scorer Type: TrueFalseInverterScorer
      • scorer_type: true_false
      • score_aggregator: OR_
        └─ Composite of 1 scorer(s):
            • Scorer Type: SelfAskRefusalScorer
            • scorer_type: true_false
            • score_aggregator: OR_
            • model_name: gpt-5.4

    ▸ Performance Metrics
      • Accuracy: 87.85%
      • Accuracy Std Error: ±0.0164
      • F1 Score: 0.8750
      • Precision: 0.8705
      • Recall: 0.8796
      • Average Score Time: 0.66s

▼ Overall Statistics
────────────────────────────────────────────────────────────────────────────────────────────────────
  📈 Summary
    • Total Strategies: 2
    • Total Attack Results: 2
    • Overall Success Rate: 0%
    • Unique Objectives: 2

▼ Per-Strategy Breakdown
────────────────────────────────────────────────────────────────────────────────────────────────────

  🔸 Strategy: baseline
    • Number of Results: 1
    • Success Rate: 0%

  🔸 Strategy: base64
    • Number of Results: 1
    • Success Rate: 0%

====================================================================================================

Strategy Composition¶

You can combine multiple converters into a single composite strategy using ScenarioCompositeStrategy. Each converter in the composite is applied in sequence.

from pyrit.scenario import ScenarioCompositeStrategy

composed = ScenarioCompositeStrategy(strategies=[FoundryStrategy.Caesar, FoundryStrategy.CharSwap])

# from pyrit.scenario import ScenarioCompositeStrategy
# composed = ScenarioCompositeStrategy(strategies=[FoundryStrategy.Caesar, FoundryStrategy.CharSwap])
# scenario_strategies = [FoundryStrategy.Base64, composed]

For more details, see the Scenarios Programming Guide and Configuration.