PyRIT Scanner - PyRIT Documentation

PyRIT (Python Risk Identification Tool for generative AI) is an open-source framework that helps security professionals proactively identify risks in generative AI systems. The scanner is the primary way to run security assessments — it executes Scenarios against a target AI system and reports results.

How It Works¶

A PyRIT scan has three key ingredients:

A Scenario — defines what to test (e.g., content harms, jailbreaks, encoding probes). Scenarios bundle attack techniques, datasets, and scoring into a reusable package.
A Target — the AI system you’re testing (e.g., an OpenAI endpoint, an Azure OpenAI deployment, a custom HTTP endpoint).
Configuration — connects the scanner to your target and registers the components it needs (targets, scorers, datasets). See Configuration.

Running Scans¶

PyRIT provides two command-line interfaces:

Tool	Best For	Documentation
`pyrit_scan`	Automated, single-command execution. CI/CD pipelines, batch processing, reproducible runs.	pyrit_scan
`pyrit_shell`	Interactive exploration. Rapid iteration, comparing results across runs, debugging scenarios.	pyrit_shell

Quick Example¶

# Run the Foundry RedTeamAgent scenario against your configured target
pyrit_scan run foundry.red_team_agent --target openai_chat --initializers target --techniques base64

Connecting to CoPyRIT¶

Point a local configuration file at the remote backend:

server:
  url: https://copyrit.example.com/
  auth_mode: auto

Then use the file without changing the default configuration in ~/.pyrit:

pyrit_scan --config-file ./.pyrit_conf list-scenarios

The CLI reads the server’s public authentication configuration. Automatic mode uses an interactive Entra device code with the exact Microsoft Graph User.Read scope and an encrypted persistent token cache. Use --auth-mode device_code to require this flow or --auth-mode none to disable authentication discovery.

--auth-mode azure_cli is an explicit compatibility mode. The Azure CLI application can issue a Graph token with permissions beyond User.Read, and the CLI sends that token to the backend. Prefer automatic device-code authentication.

Built-in Scenarios¶

PyRIT ships with scenarios organized into the following families:

Family	Scenarios	Documentation
Adaptive	TextAdaptive	Adaptive Scenarios
AIRT	RapidResponse, Psychosocial, Cyber, Jailbreak, Multilingual, Leakage, Scam	AIRT Scenarios
Benchmark	AdversarialBenchmark	Benchmark Scenarios
Foundry	RedTeamAgent	Foundry Scenarios
Garak	Encoding, FigStep, WebInjection, Doctor, SystemPromptExtraction, PackageHallucination, AudioAchillesHeel	Garak Scenarios

Each scenario page shows how to run it with minimal configuration.

For Developers¶

If you want to build custom scenarios or understand the programming model behind scenarios, see the Scenarios Programming Guide. For details on attack techniques, dataset configuration, and advanced programmatic usage, see Common Scenario Parameters.