Content Safety
GenAIScript has multiple built-in safety features to protect the system from malicious attacks.
System prompts
The following safety prompts are included by default when running a prompt, unless the system option is configured:
- system.safety_harmful_content, safety prompt against Harmful Content: Hate and Fairness, Sexual, Violence, Self-Harm. See https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/safety-system-message-templates.
- system.safety_jailbreak, safety script to ignore prompting instructions in code sections, which are created by the
def
function. - system.safety_protected_material safety prompt against Protected material. See https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/safety-system-message-templates
Other system scripts can be added to the prompt by using the system
option.
- system.safety_ungrounded_content_summarization safety prompt against ungrounded content in summarization
Azure AI Content Safety services
Azure AI Content Safety provides a set of service to protect LLM application from various attacks.
GenAIScript provides a set of APIs to interact with Azure AI Content Safety services
through the contentSafety
global object.
Configuration
Create a Content Safety resource in the Azure portal to get your key and endpoint.
Navigate to Access Control (IAM), then View My Access. Make sure your user or service principal has the Cognitive Services User role. If you get a
401
error, click on Add, Add role assignment and add the Cognitive Services User role to your user.Navigate to Resource Management, then Keys and Endpoint.
Copy the endpoint information and add it in your
.env
file asAZURE_CONTENT_SAFETY_ENDPOINT
.
Managed Identity
GenAIScript will use the default Azure token resolver to authenticate with the Azure Content Safety service.
You can override the credential resolver by setting the AZURE_CONTENT_SAFETY_CREDENTIAL
environment variable.
API Key
Copy the value of one of the keys into a AZURE_CONTENT_SAFETY_KEY
in your .env
file.
Detect Prompt Injection
The detectPromptInjection
method uses the Azure Prompt Shield
service to detect prompt injection in the given text.
The def also supports setting a detectPromptInjection
flag to apply the detection to each file.
Detect Harmful content
The detectHarmfulContent
method uses the
Azure Content Safety
to scan for harmfull content categories.