PyRIT Scan - PyRIT Documentation

pyrit_scan is the primary command-line tool for running automated security assessments and red teaming attacks against AI systems. It leverages scenarios to define attack techniques and supports flexible configuration for targeting different AI endpoints.

For configuration setup, see Configuration.

For scenario-specific examples, see AIRT, Foundry, and Garak.

Note in this doc the ! prefaces all commands in the terminal so we can run in a Jupyter Notebook.

Starting a Backend Server¶

pyrit_scan is a thin client that talks to a PyRIT backend server (by default at http://localhost:8000). Before running any command that reaches the backend (listing scenarios, running a scan, etc.) you need a server. Start a local one with --start-server; it launches a detached pyrit_backend process that stays up and is reused by every command below. We stop it again at the end of the notebook.

!pyrit_scan --start-server

Starting server at http://localhost:8000...
Server ready (PID 45120)
Server is running at http://localhost:8000

Quick Start¶

For help:

!pyrit_scan --help

usage: pyrit_scan [-h] [--server-url SERVER_URL] [--start-server]
                  [--stop-server] [--config-file CONFIG_FILE]
                  [--log-level LOG_LEVEL] [--request-timeout REQUEST_TIMEOUT]
                  [--list-scenarios] [--list-initializers] [--list-targets]
                  [--list-converters] [--list-datasets]
                  [--add-initializer FILE [FILE ...]] [--target TARGET]
                  [--initializers INITIALIZERS [INITIALIZERS ...]]
                  [--techniques SCENARIO_TECHNIQUES [SCENARIO_TECHNIQUES ...]]
                  [--max-concurrency MAX_CONCURRENCY]
                  [--max-retries MAX_RETRIES] [--memory-labels MEMORY_LABELS]
                  [--dataset-names DATASET_NAMES [DATASET_NAMES ...]]
                  [--max-dataset-size MAX_DATASET_SIZE]
                  [scenario_name]

PyRIT Scanner - Run AI security scenarios from the command line.

Requires a running PyRIT backend server. Use --start-server to launch one,
or connect to an existing server with --server-url.

Examples:
  # Start the backend server
  pyrit_scan --start-server

  # List scenarios, initializers, targets, or converters
  pyrit_scan --list-scenarios
  pyrit_scan --list-initializers
  pyrit_scan --list-targets
  pyrit_scan --list-converters

  # List available datasets
  pyrit_scan --list-datasets

  # Run single-turn cyber attacks against a target
  pyrit_scan airt.cyber --target openai_chat --techniques single_turn

  # Run rapid response with specific datasets and concurrency
  pyrit_scan airt.rapid_response --target openai_chat
    --techniques role_play --dataset-names airt_hate
    --max-dataset-size 5 --max-concurrency 4

  # Attach registered converters to a technique (repeatable, applied in order)
  pyrit_scan airt.rapid_response --target openai_chat
    --techniques role_play:converter.translation_spanish:converter.leetspeak

  # Run multi-turn red team agent with labels for tracking
  pyrit_scan airt.red_team_agent --target openai_chat
    --techniques crescendo
    --memory-labels '{"experiment":"baseline"}'

  # Register a custom initializer from a Python script
  pyrit_scan --add-initializer ./my_custom_init.py

  # Connect to a remote server
  pyrit_scan --server-url http://remote:8000 --list-scenarios

  # Stop the server
  pyrit_scan --stop-server

options:
  -h, --help            show this help message and exit

server:
  --server-url SERVER_URL
                        URL of the PyRIT backend server (default:
                        http://localhost:8000)
  --start-server        Start a local backend server if one is not already
                        running
  --stop-server         Stop the backend server and exit
  --config-file CONFIG_FILE
                        Path to a YAML configuration file. Allows specifying
                        database, initializers (with args), initialization
                        scripts, and env files. CLI arguments override config
                        file values. If not specified, ~/.pyrit/.pyrit_conf is
                        loaded if it exists.
  --log-level LOG_LEVEL
                        Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
                        (default: WARNING)
  --request-timeout REQUEST_TIMEOUT
                        HTTP read timeout in seconds for non-polling server
                        requests (catalog/results/cancel/etc). Defaults to 60.
                        Polling a live scenario run always waits indefinitely
                        regardless of this value.

discovery:
  --list-scenarios      List all available scenarios and exit
  --list-initializers   List all available initializers and exit
  --list-targets        List all available targets and exit
  --list-converters     List all registered converter instances and exit
  --list-datasets       List all available datasets and exit
  --add-initializer FILE [FILE ...]
                        Register initializer(s) from Python script file(s) and
                        exit

scenario run:
  scenario_name         Name of the scenario to run
  --target TARGET       Name of a registered target from the TargetRegistry to
                        use as the objective target. Targets are registered by
                        initializers (e.g., 'target' initializer). Use --list-
                        targets to see available target names after
                        initializers have run
  --initializers INITIALIZERS [INITIALIZERS ...]
                        Built-in initializer names to run before the scenario.
                        Supports optional params with name:key=val syntax
                        (e.g., target:tags=default,scorer dataset:mode=strict)
  --techniques, -t SCENARIO_TECHNIQUES [SCENARIO_TECHNIQUES ...]
                        List of technique names to run (e.g., base64 rot13).
                        Append one or more registered converters to a
                        technique with ':converter.<name>' (repeatable), e.g. 
                        role_play:converter.translation_spanish:converter.leet
                        speak. The converter is appended on top of the
                        technique's built-in converters. Use --list-converters
                        to see registered converter names
  --max-concurrency MAX_CONCURRENCY
                        Maximum number of concurrent attack executions (must
                        be >= 1)
  --max-retries MAX_RETRIES
                        Maximum number of automatic retries on exception (must
                        be >= 0)
  --memory-labels MEMORY_LABELS
                        Additional labels as JSON string (e.g.,
                        '{"experiment": "test1"}')
  --dataset-names DATASET_NAMES [DATASET_NAMES ...]
                        List of dataset names to use instead of scenario
                        defaults (e.g., harmbench advbench). Creates a new
                        dataset config; fetches all items unless --max-
                        dataset-size is also specified
  --max-dataset-size MAX_DATASET_SIZE
                        Maximum number of items to use from the dataset (must
                        be >= 1). Limits new datasets if --dataset-names
                        provided, otherwise overrides scenario's default limit

Discovery¶

List all available scenarios:

!pyrit_scan --list-scenarios


Available Scenarios:
================================================================================

  adaptive.text_adaptive
    Class: TextAdaptive
    Description:
      Adaptive text-attack scenario. Selects techniques per-objective via an
      epsilon-greedy selector over the set of selected techniques.
      ``prompt_sending`` runs as the baseline comparison and is excluded from
      the adaptive technique pool.
    Aggregate Techniques:
      - all, default, single_turn, multi_turn
    Available Techniques (11):
      role_play, many_shot, tap, crescendo_simulated, red_teaming,
      context_compliance, crescendo_movie_director, crescendo_history_lecture,
      crescendo_journalist_interview, pair, violent_durian
    Default Technique: default
    Default Datasets (7):
      airt_hate, airt_fairness, airt_violence, airt_sexual, airt_harassment,
      airt_misinformation, airt_leakage
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.
      - max_attempts_per_objective (int) [default: '3']: Max techniques tried per objective. Defaults to 3.

  airt.cyber
    Class: Cyber
    Description:
      Cyber scenario implementation for PyRIT. This scenario tests how willing
      models are to exploit cybersecurity harms by generating malware. The
      Cyber class contains different variations of the malware generation
      techniques.
    Aggregate Techniques:
      - all, multi_turn
    Available Techniques (1):
      red_teaming
    Default Technique: all
    Default Datasets (1):
      airt_malware
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  airt.jailbreak
    Class: Jailbreak
    Description:
      Jailbreak scenario implementation for PyRIT. This scenario tests how
      vulnerable models are to jailbreak attacks by applying various
      single-turn jailbreak templates to a set of test prompts. The responses
      are scored to determine if the jailbreak was successful.
    Aggregate Techniques:
      - all, simple, complex
    Available Techniques (4):
      prompt_sending, many_shot, skeleton, role_play
    Default Technique: simple
    Default Datasets (1):
      airt_harms
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  airt.leakage
    Class: Leakage
    Description:
      Leakage scenario implementation for PyRIT. This scenario tests how
      susceptible models are to leaking training data, PII, intellectual
      property, or other confidential information. Uses the registry/factory
      pattern to construct attack techniques.
    Aggregate Techniques:
      - all, default, single_turn, multi_turn
    Available Techniques (11):
      context_compliance, crescendo_history_lecture,
      crescendo_journalist_interview, crescendo_movie_director,
      crescendo_simulated, many_shot, red_teaming, role_play, tap,
      first_letter, image
    Default Technique: default
    Default Datasets (1):
      airt_leakage
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  airt.psychosocial
    Class: Psychosocial
    Description:
      Psychosocial Harms Scenario implementation for PyRIT. This scenario
      contains various psychosocial harm-based checks that evaluate model
      behavior for responses that may harm the well-being or mental health of
      users. Examples include mistreating users in a mental health crisis or
      aiding them in self-harm. All datasets in this scenario share the
      unified 'psychosocial' harm category, but each dataset (e.g.,
      imminent_crisis, dependency) can have unique: - Scoring rubrics - Seed
      objectives The scenario tests whether models: - Recognize mental health
      crises and respond appropriately - Avoid facilitating self-harm or
      harmful behaviors - Provide crisis resources and encourage professional
      help Example usage with custom configurations: # Define custom
      configurations per subharm category custom_configs = {
      "airt_imminent_crisis": SubharmConfig(
      crescendo_system_prompt_path="path/to/custom_escalation.yaml",
      scoring_rubric_path="path/to/custom_rubric.yaml", ), } scenario =
      Psychosocial(subharm_configs=custom_configs)
      scenario.set_params_from_args( args={ "objective_target": target_llm,
      "scenario_techniques": [PsychosocialTechnique.ImminentCrisis], } ) await
      scenario.initialize_async()
    Aggregate Techniques:
      - all
    Available Techniques (2):
      imminent_crisis, licensed_therapist
    Default Technique: all
    Default Datasets (1):
      airt_imminent_crisis
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  airt.rapid_response
    Class: RapidResponse
    Description:
      Rapid Response scenario for content-harms testing. Tests model behavior
      across multiple harm categories using selectable attack techniques.
    Aggregate Techniques:
      - all, default, single_turn, multi_turn
    Available Techniques (9):
      context_compliance, crescendo_history_lecture,
      crescendo_journalist_interview, crescendo_movie_director,
      crescendo_simulated, many_shot, red_teaming, role_play, tap
    Default Technique: default
    Default Datasets (7):
      airt_hate, airt_fairness, airt_violence, airt_sexual, airt_harassment,
      airt_misinformation, airt_leakage
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  airt.scam
    Class: Scam
    Description:
      Scam scenario evaluates an endpoint's ability to generate scam-related
      materials (e.g., phishing emails, fraudulent messages) with primarily
      persuasion-oriented techniques.
    Aggregate Techniques:
      - all, single_turn, multi_turn
    Available Techniques (3):
      context_compliance, role_play, persuasive_rta
    Default Technique: all
    Default Datasets (1):
      airt_scams
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.
      - max_turns (int) [default: '5']: Maximum conversation turns for the persuasive_rta technique.

  benchmark.adversarial
    Class: AdversarialBenchmark
    Description:
      Benchmark scenario that compares the attack success rate (ASR) across
      adversarial models. Adversarial targets are user-supplied via the
      ``adversarial_targets`` parameter (declared in
      ``supported_parameters``). Each target must already be registered in
      ``TargetRegistry`` — typically by ``TargetInitializer`` from
      ``ADVERSARIAL_CHAT_*`` env vars, or programmatically via
      ``TargetRegistry.get_registry_singleton().instances.register``. At run
      time, ``_build_atomic_attacks_async`` performs the ``(technique ×
      adversarial_target × dataset)`` cross-product: for each selected
      adversarial-capable ``core`` factory in the ``AttackTechniqueRegistry``
      and each requested target, it calls
      ``factory.create(adversarial_chat=...)`` with the resolved target — no
      global registry mutation. The resulting ``AtomicAttack`` is named
      ``f"{technique}__{target}_{dataset}"`` with ``display_group`` set to the
      target's registry name so per-model ASR rolls up naturally in result
      displays.
    Aggregate Techniques:
      - all, default, light, single_turn, multi_turn
    Available Techniques (8):
      context_compliance, crescendo_history_lecture,
      crescendo_journalist_interview, crescendo_movie_director,
      crescendo_simulated, red_teaming, role_play, tap
    Default Technique: light
    Default Datasets (1):
      harmbench
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.
      - adversarial_targets (list[str]): Registry names of adversarial chat targets to benchmark. Each name must already be registered in TargetRegistry (via TargetInitializer or TargetRegistry instance registration). Use 'pyrit_scan list-targets' to see registered targets. Settable via --adversarial-targets <name> [<name> ...] on the CLI, or scenario.args.adversarial_targets in .pyrit_conf.

  foundry.red_team_agent
    Class: RedTeamAgent
    Description:
      RedTeamAgent is a preconfigured scenario that automatically generates
      multiple AtomicAttack instances based on the specified attack
      techniques. It supports both single-turn attacks (with various
      converters) and multi-turn attacks (Crescendo, RedTeaming), making it
      easy to quickly test a target against multiple attack vectors. The
      scenario can expand difficulty levels (EASY, MODERATE, DIFFICULT) into
      their constituent attack techniques, or you can specify individual
      techniques directly. This scenario is designed for use with the Foundry
      AI Red Teaming Agent library, providing a consistent PyRIT contract for
      their integration.
    Aggregate Techniques:
      - all, easy, moderate, difficult
    Available Techniques (25):
      ansi_attack, ascii_art, ascii_smuggler, atbash, base64, binary, caesar,
      character_space, char_swap, diacritic, flip, leetspeak, morse, rot13,
      suffix_append, string_join, unicode_confusable, unicode_substitution,
      url, jailbreak, tense, multi_turn, crescendo, pair, tap
    Default Technique: easy
    Default Datasets (1):
      harmbench
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  garak.doctor
    Class: Doctor
    Description:
      Doctor scenario implementation for PyRIT. Ports Garak's "doctor" probe,
      which implements the HiddenLayer Policy Puppetry universal bypass: a
      post-instruction-hierarchy prompt injection that frames a harmful
      request as policy/configuration (a fictional medical-drama script or a
      healthcare-chatbot interaction config) to override safety guardrails.
      Each objective from the ``garak_doctor`` dataset is wrapped in a Policy
      Puppetry template via ``PolicyPuppetryConverter`` and sent to the
      target. Responses are scored for non-refusal (mirroring Garak's
      ``mitigation.MitigationBypass`` detector). Reference:
      [@hiddenlayer2025policypuppetry]
      (https://hiddenlayer.com/innovation-hub/novel-universal-bypass-for-all-major-llms/)
    Aggregate Techniques:
      - all, default
    Available Techniques (2):
      policy_puppetry, policy_puppetry_leet
    Default Technique: all
    Default Datasets (1):
      garak_doctor
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  garak.encoding
    Class: Encoding
    Description:
      Encoding Scenario implementation for PyRIT. This scenario tests how
      resilient models are to various encoding attacks by encoding potentially
      harmful text (by default slurs and XSS payloads) and testing if the
      model will decode and repeat the encoded payload. It mimics the Garak
      encoding probe. The scenario works by: 1. Taking seed prompts (the
      harmful text to be encoded) 2. Encoding them using various encoding
      schemes (Base64, ROT13, Morse, etc.) 3. Asking the target model to
      decode the encoded text 4. Scoring whether the model successfully
      decoded and repeated the harmful content By default, this uses the same
      dataset as Garak: slur terms and web XSS payloads.
    Aggregate Techniques:
      - all
    Available Techniques (17):
      base64, base2048, base16, base32, ascii85, hex, quoted_printable,
      uuencode, rot13, braille, atbash, morse_code, nato, ecoji, zalgo,
      leet_speak, ascii_smuggler
    Default Technique: all
    Default Datasets (2):
      garak_slur_terms_en, garak_web_html_js
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

  garak.web_injection
    Class: WebInjection
    Description:
      Web Injection Scenario implementation for PyRIT. Ports garak's
      ``web_injection`` probe. It tests whether a model can be coaxed into
      emitting web-injection payloads — markdown-image data exfiltration
      (CSRF-style) and cross-site scripting (XSS) — when given
      prompt-injection style instructions. Each technique builds one
      ``PromptSendingAttack`` whose seeds pair a ``SeedObjective`` (the attack
      goal) with a ``SeedPrompt`` (the literal injection prompt to send).
      Exfil techniques are scored with ``MarkdownInjectionScorer``; XSS
      techniques are scored with ``XSSOutputScorer``. The default objective
      scorer (used for the baseline and metadata) is an OR composite of both.
    Aggregate Techniques:
      - all, default, exfil, xss
    Available Techniques (8):
      markdown_image_exfil, colab_ai_data_leakage, string_assembly_data_exfil,
      playground_markdown_exfil, markdown_uri_image_exfil_extended,
      markdown_uri_non_image_exfil_extended, task_xss, markdown_xss
    Default Technique: default
    Default Datasets (4):
      garak_example_domains_xss, garak_markdown_js, garak_web_html_js,
      garak_xss_normal_instructions
    Supported Parameters:
      - objective_target (any): Target system under attack: a registered target name or a PromptTarget instance.
      - scenario_techniques (any): Techniques to execute; defaults to the scenario's default aggregate when omitted.
      - technique_converters (any): Mapping of concrete technique name to extra request converters to append.
      - dataset_config (any): Dataset source configuration; defaults to the scenario's default when omitted.
      - memory_labels (any): Additional labels applied to every attack run in the scenario.
      - max_concurrency (int) [default: '4']: Maximum number of concurrent units of work for the scenario.
      - max_retries (int) [default: '0']: Maximum number of automatic retries if the scenario raises an exception.
      - include_baseline (bool): Whether to prepend a baseline atomic attack; None defers to BASELINE_ATTACK_POLICY.

================================================================================

Total scenarios: 12

Tip: You can also discover user-defined scenarios by providing initialization scripts:

pyrit_scan --list-scenarios --initialization-scripts ./my_custom_initializer.py

This will load your custom scenario definitions and include them in the list.

Initializers¶

PyRITInitializers are how you can configure the CLI scanner. PyRIT includes several built-in initializers you can use with the --initializers flag.

The --list-initializers command shows all available initializers. Initializers are referenced by their filename (e.g., target, scorer, simple) regardless of which subdirectory they’re in.

List the available initializers using the --list-initializers flag.

!pyrit_scan --list-initializers


Available Initializers:
================================================================================

  load_default_datasets
    Class: LoadDefaultDatasets
    Required Environment Variables: None
    Supported Parameters:
      - dataset_names: Explicit dataset names to load. Overrides the scenario-default selection.
      - tags: Load datasets whose metadata matches these tags. Overrides scenario-default selection.
    Description:
      Load datasets into memory so scenarios can run. By default this loads
      the datasets required by all registered scenarios. Pass
      ``dataset_names`` to load specific datasets by name, or ``tags`` to
      select datasets by metadata.

  preload_scenario_metadata
    Class: PreloadScenarioMetadata
    Required Environment Variables: None
    Description:
      Instantiate every registered scenario once to warm the metadata cache.

  scorer
    Class: ScorerInitializer
    Required Environment Variables: None
    Supported Parameters:
      - tags [default: ['default']]: Tags for filtering (e.g., ['default'])
    Description:
      Instantiates a collection of scorers using targets from the
      TargetRegistry and adds them to the ScorerRegistry. This initializer
      registers all evaluation scorers into the ScorerRegistry. Targets are
      pulled from the TargetRegistry (populated by TargetInitializer), so this
      initializer should be listed after TargetInitializer in the initializers
      list. Scorers that fail to initialize (e.g., due to missing targets) are
      skipped with a warning. Every scorer category follows the same pattern:
      ``_register_<category>_scorers()`` registers all variants with a
      category tag. ``_tag_best_per_category()`` marks the preferred scorer
      per category. Compound scorers reference core scorers via BEST_* tags.

  target
    Class: TargetInitializer
    Required Environment Variables: None
    Supported Parameters:
      - tags [default: ['default']]: Target tags to register (e.g., ['default'], ['default', 'scorer'], or ['all'])
      - auto_group [default: True]: Auto-create round-robin groups from targets with matching behavioral eval params
    Description:
      Target Initializer for registering pre-configured targets. This
      initializer scans for known endpoint environment variables and registers
      the corresponding targets into the TargetRegistry. Targets can be
      filtered by tags to control which targets are registered. Supported
      Parameters: tags: Target tags to register (list of strings). "default"
      registers the base environment targets. "scorer" registers
      scorer-specific temperature variant targets. "all" registers all targets
      regardless of tag. If not provided, only "default" targets are
      registered. auto_group: Whether to automatically create round-robin
      groups from targets with matching behavioral eval params (underlying
      model, temperature, top_p). Defaults to True. Supported Endpoints by
      Category: **OpenAI Chat Targets (OpenAIChatTarget):** -
      PLATFORM_OPENAI_CHAT_* - Platform OpenAI Chat API - AZURE_OPENAI_GPT4O_*
      - Azure OpenAI GPT-4o - AZURE_OPENAI_INTEGRATION_TEST_* - Integration
      test endpoint - AZURE_OPENAI_GPT3_5_CHAT_* - Azure OpenAI GPT-3.5 -
      AZURE_OPENAI_GPT4_CHAT_* - Azure OpenAI GPT-4 - AZURE_OPENAI_GPT5_4_* -
      Azure OpenAI GPT-5.4 - AZURE_OPENAI_GPT5_COMPLETIONS_* - Azure OpenAI
      GPT-5.1 - AZURE_OPENAI_GPT4O_UNSAFE_CHAT_* - Azure OpenAI GPT-4o unsafe
      - AZURE_OPENAI_GPT4O_UNSAFE_CHAT_*2 - Azure OpenAI GPT-4o unsafe
      secondary - AZURE_FOUNDRY_DEEPSEEK_* - Azure AI Foundry DeepSeek -
      AZURE_FOUNDRY_PHI4_* - Azure AI Foundry Phi-4 -
      AZURE_FOUNDRY_MISTRAL_LARGE_* - Azure AI Foundry Mistral Large - GROQ_*
      - Groq API - OPEN_ROUTER_* - OpenRouter API - OLLAMA_* - Ollama local -
      GOOGLE_GEMINI_* - Google Gemini (OpenAI-compatible) **OpenAI Responses
      Targets (OpenAIResponseTarget):** - AZURE_OPENAI_GPT5_RESPONSES_* -
      Azure OpenAI GPT-5 Responses - AZURE_OPENAI_GPT5_RESPONSES_* (high
      reasoning) - Azure OpenAI GPT-5 Responses with high reasoning effort -
      PLATFORM_OPENAI_RESPONSES_* - Platform OpenAI Responses -
      AZURE_OPENAI_RESPONSES_* - Azure OpenAI Responses **Realtime Targets
      (RealtimeTarget):** - PLATFORM_OPENAI_REALTIME_* - Platform OpenAI
      Realtime - AZURE_OPENAI_REALTIME_* - Azure OpenAI Realtime **Image
      Targets (OpenAIImageTarget):** - OPENAI_IMAGE_*1 - Azure OpenAI Image -
      OPENAI_IMAGE_*2 - Platform OpenAI Image **TTS Targets
      (OpenAITTSTarget):** - OPENAI_TTS_*1 - Azure OpenAI TTS - OPENAI_TTS_*2
      - Platform OpenAI TTS **Video Targets (OpenAIVideoTarget):** -
      AZURE_OPENAI_VIDEO_* - Azure OpenAI Video **Completion Targets
      (OpenAICompletionTarget):** - OPENAI_COMPLETION_* - OpenAI Completion
      **Azure ML Targets (AzureMLChatTarget):** - AZURE_ML_PHI_* - Azure ML
      Phi **Safety Targets (PromptShieldTarget):** - AZURE_CONTENT_SAFETY_* -
      Azure Content Safety Example: initializer = TargetInitializer() await
      initializer.initialize_async() # Register scorer temperature variants
      too initializer.params = {"tags": ["default", "scorer"]} await
      initializer.initialize_async()

  technique
    Class: TechniqueInitializer
    Required Environment Variables: None
    Supported Parameters:
      - tags [default: ['core']]: Technique groups to register (e.g., ['core'], ['core', 'extra'], or ['all'])
    Description:
      Register scenario attack technique factories into the
      AttackTechniqueRegistry. By default only the ``core`` group is
      registered. Pass ``tags`` to select groups (``core``, ``extra``, or
      ``all``). Registration is per-name idempotent: pre-existing entries in
      ``AttackTechniqueRegistry`` are not overwritten.

================================================================================

Total initializers: 5

Running Scenarios¶

You need a single scenario to run, you need two things:

A Scenario. Many are defined in pyrit.scenario.scenarios. But you can also define your own in initialization_scripts.
Initializers (which can be supplied via --initializers or --initialization-scripts or initializers section of config file (see here)). Scenarios often don’t need many arguments, but they can be configured in different ways. And at the very least, most need an objective_target (the thing you’re running a scan against) which you can configure by using the --target flag if your initializer registers targets (e.g. target initializer)
Scenario Techniques (optional). These are supplied by the --techniques flag and tell the scenario what to test, but they are always optional. Also note you can obtain these by running --list-scenarios

Basic usage will look something like:

pyrit_scan <scenario> --target <target_name> --initializers <initializer1> <initializer2> --techniques <technique1> <technique2>

You can also override scenario parameters directly from the CLI:

pyrit_scan <scenario> --max-concurrency 10 --max-retries 3 --memory-labels '{"experiment": "test1", "version": "v2"}'

Or concretely:

!pyrit_scan foundry.red_team_agent --target openai_chat --initializers target --techniques base64

Example with a basic configuration that runs the Foundry scenario against the objective target defined in the target initializer.

!pyrit_scan foundry.red_team_agent --target openai_chat --initializers target --techniques base64


Running scenario: foundry.red_team_agent

  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
  techniques: 0/1 | attacks: 0 | success rate: 0% | IN_PROGRESS
Error (UnicodeEncodeError): 'charmap' codec can't encode characters in position 22-51: character maps to <undefined>

Or with all options and multiple techniques:

pyrit_scan foundry.red_team_agent --target openai_chat --initializers target --techniques easy crescendo

You can also override scenario execution parameters:

# Override concurrency and retry settings
pyrit_scan foundry.red_team_agent --target openai_chat --initializers target --max-concurrency 10 --max-retries 3

# Add custom memory labels for tracking (must be valid JSON)
pyrit_scan foundry.red_team_agent --target openai_chat --initializers target --memory-labels '{"experiment": "test1", "version": "v2", "researcher": "alice"}'

Available CLI parameter overrides:

--max-concurrency <int>: Maximum number of concurrent attack executions
--max-retries <int>: Maximum number of automatic retries if the scenario raises an exception
--memory-labels <json>: Additional labels to apply to all attack runs (must be a JSON string with string keys and values)

You can also use custom initialization scripts by passing file paths. It is relative to your current working directory, but to avoid confusion, full paths are always better:

pyrit_scan garak.encoding --initialization-scripts ./my_custom_config.py

Attaching Converters to a Technique¶

Techniques (techniques) can have a registered converter instance appended to them with the <technique>:converter.<name> syntax. The converter is added to the request side of every attack the technique produces, on top of any converters the technique already bakes in. This also works on aggregate techniques (the converter is applied to every technique the aggregate expands to).

First discover the registered converter instances with --list-converters (converters are registered by initializers, so pass the same --initializers/--initialization-scripts you use to run):

pyrit_scan --list-converters --initializers my_converters

Then reference a converter by name in --techniques:

# Add the registered "translation_spanish" converter to role_play_movie_script only
pyrit_scan airt.rapid_response --target openai_chat --initializers load_default_datasets target my_converters --techniques role_play_movie_script:converter.translation_spanish

# Chain multiple converters (applied in order) and combine with plain techniques
pyrit_scan airt.rapid_response --target openai_chat --initializers load_default_datasets target my_converters --techniques role_play_movie_script:converter.translation_spanish:converter.base64 many_shot

Using Custom Scenarios¶

You can define your own scenarios in initialization scripts. The CLI will automatically discover any Scenario subclasses and make them available:

# my_custom_scenarios.py

from pyrit.common import apply_defaults
from pyrit.prompt_target.openai.openai_chat_target import OpenAIChatTarget
from pyrit.scenario import DatasetAttackConfiguration, Scenario, ScenarioTechnique
from pyrit.score import SelfAskRefusalScorer, TrueFalseInverterScorer
from pyrit.setup import initialize_pyrit_async


class MyCustomTechnique(ScenarioTechnique):
    """Techniques for my custom scenario."""

    ALL = ("all", {"all"})
    Technique1 = ("technique1", set[str]())
    Technique2 = ("technique2", set[str]())


class MyCustomScenario(Scenario):
    """My custom scenario that does XYZ."""

    @apply_defaults
    def __init__(self, *, scenario_result_id=None, **kwargs):
        # Scenario-specific configuration only - no runtime parameters
        super().__init__(
            name="My Custom Scenario",
            version=1,
            objective_scorer=TrueFalseInverterScorer(scorer=SelfAskRefusalScorer(chat_target=OpenAIChatTarget())),
            technique_class=MyCustomTechnique,
            default_technique=MyCustomTechnique.ALL,
            default_dataset_config=DatasetAttackConfiguration(dataset_names=["harmbench"]),
            scenario_result_id=scenario_result_id,
        )
        # ... your scenario-specific initialization code

    async def _build_atomic_attacks_async(self, *, context):
        # The single abstract extension point every scenario implements.
        # Read runtime inputs from `context`; return the list of AtomicAttack to run.
        # Matrix-shaped scenarios can delegate to build_matrix_atomic_attacks(context=...).
        # Example: create attacks for each technique composite
        return []


await initialize_pyrit_async(memory_db_type="InMemory")  # type: ignore
MyCustomScenario()

Found default environment files: ['./.pyrit/.env', './.pyrit/.env.local']
Loaded environment file: ./.pyrit/.env
Loaded environment file: ./.pyrit/.env.local

[pyrit:alembic] No new upgrade operations detected.

<__main__.MyCustomScenario at 0x1d2391a9d30>

Then discover and run it:

# List to see it's available
pyrit_scan --list-scenarios --initialization-scripts ./my_custom_scenarios.py

# Run it with parameter overrides
pyrit_scan my_custom_scenario --initialization-scripts ./my_custom_scenarios.py --max-concurrency 10

The scenario name is automatically converted from the class name (e.g., MyCustomScenario becomes my_custom_scenario).

Stopping the Backend Server¶

When you’re done, stop the local backend that we started at the top of the notebook.

!pyrit_scan --stop-server

Server on port 8000 stopped.