pyrit.analytics

Analytics module for PyRIT conversation and result analysis.

Functions¶

analyze_results¶

analyze_results(attack_results: list[AttackResult]) → dict[str, AttackStats | dict[str, AttackStats]]

Analyze a list of AttackResult objects and return overall and grouped statistics.

Returns:

• dict[str, AttackStats | dict[str, AttackStats]] — A dictionary of AttackStats objects. The overall stats are accessible with the key

• dict[str, AttackStats | dict[str, AttackStats]] — “Overall”, and the stats of any attack can be retrieved using “By_attack_identifier”

• dict[str, AttackStats | dict[str, AttackStats]] — followed by the identifier of the attack.

Raises:

• ValueError — if attack_results is empty.

• TypeError — if any element is not an AttackResult.

get_cached_results_for_technique¶

get_cached_results_for_technique(memory_interface: MemoryInterface, technique_eval_hash: str, objective_target_eval_hash: str, additional_filters: Optional[Sequence[IdentifierFilter]] = None) → list[AttackResult]

Return cached AttackResults matching a (technique × objective target) pair.

Memory is queried for AttackResults whose stamped atomic_attack_identifier.eval_hash equals technique_eval_hash, then results are filtered in Python to those whose nested objective target produces the requested objective_target_eval_hash (computed via ObjectiveTargetEvaluationIdentifier). Returned results are sorted newest-first by timestamp so the most recent is at index 0.

No scenario scoping is applied; this is a behavioral cache spanning every run that produced the same (technique × target) combination. Callers that need scenario-level scoping should pass additional IdentifierFilters or filter the returned list themselves.

Returns:

• list[AttackResult] — list[AttackResult]: Matching attack results sorted newest-first. Empty list if no cache hit.

ApproximateTextMatching¶

Bases: TextMatching

Approximate text matching using n-gram overlap.

This strategy computes the proportion of character n-grams from the target that are present in the text. Useful for detecting partial matches, encoded content, or text with variations.

Constructor Parameters:

Methods:

get_overlap_score¶

get_overlap_score(target: str, text: str) → float

Get the n-gram overlap score without threshold comparison.

Useful for getting detailed scoring information.

Returns:

• float — The n-gram overlap score between 0.0 and 1.0.

is_match¶

is_match(target: str, text: str) → bool

Check if target approximately matches text using n-gram overlap.

Returns:

• bool — True if n-gram overlap score exceeds threshold, False otherwise.

AttackStats¶

Statistics for attack analysis results.

ConversationAnalytics¶

Handles analytics operations on conversation data, such as finding similar chat messages based on conversation history or embedding similarity.

Constructor Parameters:

Methods:

get_prompt_entries_with_same_converted_content¶

get_prompt_entries_with_same_converted_content(chat_message_content: str) → list[ConversationMessageWithSimilarity]

Retrieve chat messages that have the same converted content.

Returns:

• list[ConversationMessageWithSimilarity] — list[ConversationMessageWithSimilarity]: A list of ConversationMessageWithSimilarity objects representing

• list[ConversationMessageWithSimilarity] — the similar chat messages based on content.

get_similar_chat_messages_by_embedding¶

get_similar_chat_messages_by_embedding(chat_message_embedding: list[float], threshold: float = 0.8) → list[EmbeddingMessageWithSimilarity]

Retrieve chat messages that are similar to the given embedding based on cosine similarity.

Returns:

• list[EmbeddingMessageWithSimilarity] — List[ConversationMessageWithSimilarity]: A list of ConversationMessageWithSimilarity objects representing

• list[EmbeddingMessageWithSimilarity] — the similar chat messages based on embedding similarity.

ExactTextMatching¶

Bases: TextMatching

Exact substring matching strategy.

Checks if the target string is present in the text as a substring.

Constructor Parameters:

Methods:

is_match¶

is_match(target: str, text: str) → bool

Check if target string is present in text.

Returns:

• bool — True if target is found in text, False otherwise.

TextMatching¶

Bases: Protocol

Protocol for text matching strategies.

Classes implementing this protocol must provide an is_match method that checks if a target string matches text according to some strategy.

Methods:

is_match¶

is_match(target: str, text: str) → bool

Check if target matches text according to the strategy.

Returns:

• bool — True if target matches text according to the strategy, False otherwise.