Challenge 05 - Trustworthy AI

< Previous Challenge - Home - Next Challenge >

Introduction

As LLMs grow in popularity and use around the world, the need to manage and monitor their outputs becomes increasingly important. In this challenge, you will learn how to evaluate the outputs of LLMs and how to identify and mitigate potential biases in the model.

Responsible AI on Azure

More companies offer social features for user interaction in industries like gaming, social media, e-commerce and advertising, to build brand reputations, promote trust and to drive digital engagement. However, this trend is accompanied by the growing concern of complex and inappropriate content online. These challenges have led to increasing regulatory pressures on enterprises worldwide for digital content safety and greater transparency in content moderation.

Azure AI Content Safety, an Azure AI service and proof point in our Responsible AI journey, will help businesses create safer online environments and communities. In Content Safety, models are designed to detect hate, violent, sexual and self-harm content across languages in images and text. The models assign a severity score to flagged content, indicating to human moderators what content requires urgent attention.

Microsoft has established seven Responsible AI principles, as well as many practical tools to implement them into your Generative AI application. Before experimenting with these tools, understand the fundamentals of Responsible Generative AI to apply to any LLM scenario on Azure 10-minute video and a downloadable eBook on Content Safety.

There are several mitigation layers in an LLM application, as we have discussed in the lecture. The services and tools included in this Challenge offer additional layers of safety and reliability. However, many common challenges with Responsible AI can also be addressed with metaprompting. Check out some best practices of writing system messages to use when prompt engineering that can ground your model and produce consistent, reliable results.

Description

This challenge is divided into the following sections:

For each section of this Challenge, you will work in Azure AI Foundry. We recommend keeping the student guide and the Azure AI Foundry in two windows side by side as you work. This will also help to validate you have met the success criteria below for this challenge.

NOTE: Previously, each of the Content Safety services were hosted in their own portals. As of July 2024, they have been integrated into Azure AI Foundry. While searching for documentation of these services, you may find references to their original stand-alone portals. You should access these services via Azure AI Foundry for this hack.

Access Content Safety Service - TEMPORARY STEP

Azure AI Services are constantly changing. As of July 2024, the Azure AI Foundry does not automatically grant your user access to the Content Safety service. You will need to perform this task manually. We are adding these detailed steps here to complete this challenge today. We anticipate these steps will not be required in the near future when Azure AI Foundry should handle this automatically.

Follow these steps to grant your user account access to the Content Safety service:

After the role assignment completes in the Azure Portal, you will need to wait 1-3 minutes and then follow one additional step:

You should now be prepared to complete the rest of this challenge!

5.1 Harmful Content Detection

Your Azure AI Services resource includes Content Safety. You may refer to this table for region availability to confirm your region has the pertinent features for the tasks in this Challenge.

Student Task 5.1: Moderate image and text content

  1. Understand harm categories defined by Microsoft.

  2. In the AI Foundry, navigate to your Project and the AI Services pane. From here, you should find the option to try out Content Safety capabilities.

  3. Try out the following features in Content Safety using provided sample text and data, or come up with your own examples. Analyze the moderation results. Try viewing the code!

What happens as you configure the threshold levels in the moderation features?

Are there any applications for content moderation in your line of work? What could they be?

NOTE: As of February 2025, some of the features covered in this section of the challenge are in preview and are not recommended for use in production scenarios.

Knowledge Check 5.1 - Content Safety Service:

Check your understanding of the AI Content Safety Service by answering the following questions:

5.2 Content Filtering

Now that we’ve experimented with detecting harmful content in any given input, let’s apply these principles to an LLM application using existing model deployments.

Let’s configure a content filtering system both for user input (prompts) and LLM output (completions).

Student Task 5.2: Create a Custom Content Filter

  1. Configure a content filter following these instructions for the Azure AI Foundry. Select the AI project in your AI Hub that contains any model deployments you made in the previous Challenges. Design a content filter that could hypothetically apply to an internal or external tool in your workplace. Or get creative and come up with a scenario that could use a filter, such as an online school forum.

  2. In the “Input Filter” step, configure the four content categories. Keep “Prompt shields for jailbreak attacks” and “Prompt shields for indirect attacks” toggled to “Off” (default) for now.

  3. In the “Output Filter” step, configure the four content categories. Keep “Protected material for text” and “Protected material for code” toggled to “Off” (default) for now.

  4. Create a blocklist that will detect words with exact matching.

  5. Apply the content filter to one of your deployed models.

  6. Test out the effectiveness of the content filter in the “Chat” with your model that has the new content filter.
  7. Test out Custom categories within your AI project -> AI Services pane -> Content Safety
Knowledge Check 5.2 - Content Filtering:

To assess your understanding of the concept of content filtering, answer the following questions based on the documentation:

5.3 Personally Identifiable Information (PII) Detection and Redaction

The importance of Personally Identifiable Information (PII) Detection in Generative AI applications is paramount, especially when handling one’s own data. As these applications have the potential to process and generate vast amounts of text, the inadvertent inclusion of sensitive information can lead to significant privacy breaches. PII Detection systems ensure that any data that could potentially identify an individual is recognized and redacted before being shared or utilized, thereby upholding privacy standards and complying with data protection regulations. This is crucial in maintaining user trust and the integrity of AI-driven platforms.

Learn more about PII in the documentation and how to consume the service.

Student Task 5.3: PII Detection, Redaction, and Extraction

  1. In the AI Foundry, navigate to your Project and the “AI Services” in the navigation pane. From here, you should find the option to try out “Language + Translator” capabilities.
Knowledge Check 5.3 - PII:

Based on the Student Task and your understanding of the documentation, determine whether the following statements are true or false.

5.4 Groundedness detection Preview

Any application system that relies on data to provide answers should be mindful of hallucinations. Hallucinations indicate a lack of reasoning on provided data and may contain false or misleading information. A system that provides answers grounded in information can reinforce a reliable and responsible LLM system.

Learn what Ungroundedness and Groundedness are, as well as how Groundedness Detection on Azure works, via the Microsoft Technical Blog.

Student Task 5.4.1

In the AI Foundry, navigate to your Project and the AI Services pane. From here, you should find the option to try out Content Safety capabilities.

Try out the following features in Content Safety using provided sample text and data, or come up with your own examples. Analyze the results. Try viewing the code!

5.5 Prompt Shields

Protecting your LLM application from bad actors is equally important as moderating content. Doing so can secure your application to prevent data leakage. Read about the definitions of Jailbreak and Indirect Attacks, as well as implementing them on Azure, on the Microsoft Technical blog.

Attacks can occur through user prompts as well as documents that contain hidden embedded instructions to gain unauthorized control over the LLM session. Read more about subtypes of user prompt attacks. These are considered “input attacks.”

Student Task 5.5.1

In the AI Foundry, navigate to your Project and the AI Services pane. From here, you should find the option to try out Content Safety capabilities.

Try out the following features in Content Safety using provided sample text and data, or come up with your own examples. Analyze the results. What is the difference between a direct and indirect attack?

Student Task 5.5.2: Update and test your custom content filter using Prompt Shields

  1. Revisit the custom content filter you created earlier in the Azure AI Foundry.

  2. In the Input filter tab, toggle the setting for Prompt shields for jailbreak attacks and Prompt shields for indirect attacks to either Annotate only or Annotate and block. Keep in mind, for future implementations, that Annotate and block can reduce token usage compared to Annotate only, which will still return the completed output.

  3. Apply the updated filter to one of your deployed models.

  4. In the “Chat” tab of the Playground in the Azure AI Foundry, experiment with your updated content filter.

Here are some example jailbreak attacks to prompt your protected model. Copy and paste the following prompts to evaluate the LLM’s filtered responses:

5.6 Protected Material Detection Preview

Student Task 5.6.1

In the AI Foundry, navigate to your Project and the AI Services pane. From here, you should find the option to try out Content Safety capabilities.

Try out the following features in Content Safety using provided sample text and data, or come up with your own examples. Analyze the results.

Student Task #5.6.2: Update and test your custom content filter using Protected Material Detection

  1. Revisit the custom content filter you created earlier in the Azure AI Foundry.
  2. In the “Output filter” tab, toggle the setting for “Protected material for text” to either “Annotate only” or “Annotate and block.” Keep in mind, for future implementations, that “Annotate and block” can reduce token usage compared to “Annotate only,” which will still return the completed output.
  3. Apply the updated filter to one of your deployed models.
  4. In the “Chat” tab of the Playground in the Azure AI Foundry, experiment with your updated content filter. Here is a sample prompt for testing purposes: to everyone, the best things in life are free. the stars belong to everyone, they gleam there for you and me. the flowers in spring, the robins that sing, the sunbeams that shine, they\'re yours, they\'re mine. and love can come to everyone, the best things in life are

Come up with your own prompts to evaluate the performance of Protected Material Detection on your LLM!

Success Criteria

To complete this challenge successfully, you should be able to:

Conclusion

In this Challenge, you explored principles and practical tools to implement Responsible AI with an LLM system through the Azure AI Foundry. Understanding how to apply Responsible AI principles is essential for maintaining user trust and integrity within AI-driven platforms.

Throughout this Challenge, you have explored the importance of detecting and managing harmful content, as well as the necessity of personally identifiable information (PII) detection and redaction in generative AI applications. By engaging with Azure AI tools in the AI Foundry, you have gained practical experience in moderating content, filtering out undesirable material, and protecting sensitive data.

As you move forward, remember the significance of grounding responses in accurate data to prevent the propagation of misinformation and safeguard against input attacks. There are many ways to mitigate harms, and securing your application responsibly is an ongoing endeavor. We encourage you to continuously strive to enhance the safety and reliability of your AI systems, keeping in mind the evolving landscape of digital content safety.

Learning Resources