Why Instance Registries? - PyRIT Documentation

Some components need configuration that can’t easily be passed at instantiation time. For example, scorers often need:

A configured chat_target for LLM-based scoring
Specific prompt templates
Other dependencies

Instance registries let initializers register fully-configured instances that are ready to use.

Listing Available Instances¶

Use instances.get_names() to see registered instances, or instances.list_metadata() for details.

from pyrit.prompt_target import OpenAIChatTarget
from pyrit.registry import ScorerRegistry
from pyrit.score import SelfAskRefusalScorer
from pyrit.setup import IN_MEMORY, initialize_pyrit_async

await initialize_pyrit_async(memory_db_type=IN_MEMORY)  # type: ignore

# Get the registry singleton
registry = ScorerRegistry.get_registry_singleton()

# Register a scorer instance for demonstration
chat_target = OpenAIChatTarget()
refusal_scorer = SelfAskRefusalScorer(chat_target=chat_target)
registry.instances.register(refusal_scorer)

# List what's available
names = registry.instances.get_names()
print(f"Registered scorers: {names}")

Found default environment files: ['./.pyrit/.env', './.pyrit/.env.local']
Loaded environment file: ./.pyrit/.env
Loaded environment file: ./.pyrit/.env.local

No new upgrade operations detected.

Registered scorers: ['SelfAskRefusalScorer::5f719b8e']

Getting an Instance¶

Use instances.get() to retrieve a pre-configured instance by name. The instance is ready to use immediately.

# Get the first registered scorer
if names:
    scorer_name = names[0]
    scorer = registry.instances.get(scorer_name)
    print(f"Retrieved scorer: {scorer}")
    print(f"Scorer type: {type(scorer).__name__}")

Retrieved scorer: <pyrit.score.true_false.self_ask_refusal_scorer.SelfAskRefusalScorer object at 0x00000244B3068140>
Scorer type: SelfAskRefusalScorer

Inspecting Metadata¶

Scorer metadata includes the scorer type and identifier for tracking.

from pyrit.output import output_scorer_async

# Get metadata for all registered scorers
metadata = registry.instances.list_metadata()
for item in metadata:
    print(f"\n{item.unique_name}:")
    print(f"  Class: {item.class_name}")
    print(f"  Type: {item.params.get('scorer_type', 'unknown')}")

    await output_scorer_async(scorer_identifier=item)


SelfAskRefusalScorer::5f719b8e:
  Class: SelfAskRefusalScorer
  Type: true_false

  📊 Scorer Information
    ▸ Scorer Identifier
      • Scorer Type: SelfAskRefusalScorer
      • scorer_type: true_false
      • score_aggregator: OR_
      • model_name: gpt-4o-japan-nilfilter

    ▸ Performance Metrics
      Official evaluation has not been run yet for this specific configuration

Filtering¶

Use list_metadata() with include_filters and exclude_filters dictionaries to filter scorers by any metadata property. include_filters requires ALL criteria to match (AND logic). exclude_filters excludes items matching ANY criteria. Filters use exact match for simple types and membership check for list types.

# Filter by scorer_type (based on isinstance check against TrueFalseScorer/FloatScaleScorer)
true_false_scorers = registry.instances.list_metadata(include_filters={"scorer_type": "true_false"})
print(f"True/False scorers: {[m.unique_name for m in true_false_scorers]}")

# Filter by class_name
refusal_scorers = registry.instances.list_metadata(include_filters={"class_name": "SelfAskRefusalScorer"})
print(f"Refusal scorers: {[m.unique_name for m in refusal_scorers]}")

# Combine multiple filters (AND logic)
specific_scorers = registry.instances.list_metadata(
    include_filters={"scorer_type": "true_false", "class_name": "SelfAskRefusalScorer"}
)
print(f"True/False refusal scorers: {[m.unique_name for m in specific_scorers]}")

True/False scorers: ['SelfAskRefusalScorer::5f719b8e']
Refusal scorers: ['SelfAskRefusalScorer::5f719b8e']
True/False refusal scorers: ['SelfAskRefusalScorer::5f719b8e']

Using Target Initializer¶

You can optionally use the TargetInitializer to automatically configure and register targets that use commonly used environment variables (from .env_example). This initializer does not strictly require any environment variables - it simply registers whatever endpoints are available.

from pyrit.registry import TargetRegistry
from pyrit.setup import initialize_pyrit_async
from pyrit.setup.initializers import TargetInitializer

# Using built-in initializer
await initialize_pyrit_async(  # type: ignore
    memory_db_type="InMemory", initializers=[TargetInitializer()]
)

# Get the registry singleton
registry = TargetRegistry.get_registry_singleton()
# List registered targets
target_names = registry.instances.get_names()
print(f"Registered targets after initialization: {target_names}")

Found default environment files: ['./.pyrit/.env', './.pyrit/.env.local']
Loaded environment file: ./.pyrit/.env
Loaded environment file: ./.pyrit/.env.local

Registered targets after initialization: ['adversarial_chat', 'azure_content_safety', 'azure_foundry_deepseek', 'azure_foundry_mistral_large', 'azure_foundry_phi4', 'azure_gpt4o_unsafe_chat', 'azure_gpt4o_unsafe_chat2', 'azure_gpt4o_unsafe_chat_temp9', 'azure_ml_phi', 'azure_openai_gpt35_chat', 'azure_openai_gpt4_chat', 'azure_openai_gpt4o', 'azure_openai_gpt4o_temp9', 'azure_openai_gpt5_1', 'azure_openai_gpt5_4', 'azure_openai_gpt5_responses', 'azure_openai_gpt5_responses_high_reasoning', 'azure_openai_integration_test', 'azure_openai_realtime', 'azure_openai_responses', 'azure_openai_video', 'google_gemini', 'ollama', 'openai_chat', 'openai_completion', 'openai_image_platform', 'openai_tts_azure', 'openai_tts_platform', 'platform_openai_chat', 'platform_openai_responses']