PyRIT — Python Risk Identification Tool

PyRIT

Python Risk Identification Tool¶

Automated and human-led AI red teaming — a flexible, extensible framework for assessing the security and safety of generative AI systems at scale.

What PyRIT Offers

Key Capabilities¶

🎯 Automated Red Teaming

Run multi-turn attack strategies like Crescendo, TAP, and Skeleton Key against AI systems with minimal setup. Single-turn and multi-turn attacks supported out of the box.

📦 Scenario Framework

Run standardized evaluation scenarios at large scale — covering content harms, psychosocial risks, data leakage, and more. Compose strategies and datasets for repeatable, comprehensive assessments across hundreds of objectives.

🖥️ CoPyRIT

A graphical user interface for human-led red teaming. Interact with AI systems directly, track findings, and collaborate with your team — all from a modern web UI.

🔌 Any Target

Test OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP endpoints or WebSockets, web app targets with Playwright, or build your own with a simple interface.

💾 Built-in Memory

Track all conversations, scores, and attack results with SQLite or Azure SQL. Export, analyze, and share results with your team.

📊 Flexible Scoring

Evaluate AI responses with true/false, Likert scale, classification, and custom scorers — powered by LLMs, Azure AI Content Safety, or your own logic.

Getting Started

Setup and Installation¶

Install PyRIT and verify installation.
For more details and alternative installation methods, see the Install PyRIT page

Roakey raising a spyglass to scan the horizon

# note: for local installation, python version 3.13 is recommended: https://www.python.org/downloads/latest/python3.13
pip install pyrit
python -c "import pyrit; print(f'PyRIT version installed: {pyrit.__version__}')"

Create and populate endpoint and startup configuration files in ~/.pyrit/.env and ~/.pyrit/.pyrit_conf with minimal content below.
For more details, see the Configure PyRIT page.

🔑 ~/.pyrit/.env

# example OPENAI_CHAT_ENDPOINT values:
# "https://api.openai.com/v1"
# "https://<project>.cognitiveservices.azure.com/openai/v1/"
# "https://<project>.services.ai.azure.com/openai/v1"
OPENAI_CHAT_ENDPOINT="<open-ai-chat-endpoint>"
OPENAI_CHAT_KEY="<your-api-key>"
OPENAI_CHAT_MODEL="<model-name>"

📄 ~/.pyrit/.pyrit_conf

memory_db_type: in_memory

initializers:
  - name: target
    args:
      tags:
        - default
        - scorer
  - name: scorer

Use PyRIT in any mode that best fits your use case: Scanner, GUI, or Framework.

🔍 Scanner

🖥️ GUI

🧩 Framework

Run security assessments from the command line with pyrit_scan or the interactive pyrit_shell. Execute built-in scenarios against your AI targets.

pyrit_scan run airt.scam --target openai_chat

Open the Scanner walkthrough directly.

Use pyrit_scan --help to learn more about what else pyrit_scan can do. For more details, see the Scanner page.

Use CoPyRIT’s graphical interface for interactive red teaming. Chat with AI systems, track findings, and collaborate with your team.

Start the local web app and give it a try:

pyrit_backend # serves webapp on http://localhost:8000/

Open the CoPyRIT GUI walkthrough directly.

For more details, see the GUI page.

Dive into PyRIT’s modular components — targets, converters, scorers, memory, and more. Create custom attacks and extend the framework.

from pyrit.executor.attack import PromptSendingAttack
from pyrit.output.attack_result.pretty import PrettyAttackResultMemoryPrinter
from pyrit.prompt_target import OpenAIChatTarget
from pyrit.setup import IN_MEMORY, initialize_pyrit_async

await initialize_pyrit_async(memory_db_type=IN_MEMORY)

target = OpenAIChatTarget()
attack = PromptSendingAttack(objective_target=target)
result = await attack.execute_async(objective="What model exactly are you? be concise.")

printer = PrettyAttackResultMemoryPrinter()
await printer.write_async(result)

For more details, see the Framework page.

Builders, contributors, consumers

Ecosystem¶

PyRIT is part of a growing community of GenAI security tools, programs, and research. Whether you’re shipping AI products, hunting vulnerabilities, or studying the space, here’s where to plug in.

🧩 Built on PyRIT¶

RAMPART (Microsoft)

Pytest-native safety & security testing for agentic AI. Turn red-team findings into repeatable CI regression tests on top of PyRIT.

AI Red Teaming Agent (Azure AI Foundry)

Managed Azure service that runs PyRIT-powered scans against models and agents deployed in Foundry — easy to get started, Foundry provides attack data and model as well as scoring.

🤝 Programs & benchmarks¶

0DIN (Mozilla)

AI Bug Bounty and Security Research Program providing AI threat feeds and security tools.

MSRC (Microsoft)

Found an AI-related vulnerability in a Microsoft product? Report it to the Microsoft Security Response Center for coordinated disclosure.

MLCommons (Jailbreak Benchmark)

MLCommons’ AI Risk & Reliability group is building a continuously refreshed jailbreak benchmark, organized by its Jailbreak Taxonomy. PyRIT’s attack libraries and prompt converters provide the implementation framework.

📚 From the Microsoft AI Red Team¶

Guidance, methodology, threat models, and learning resources for AI red teaming — straight from the team that builds PyRIT. Browse the full Microsoft AI Red Team hub on Microsoft Learn, the Bibliography, or explore a featured resource: