Skip to main content

Manage Sensitivity Labels for Agent Embedded File Content

Implementation Effort: Medium – Requires reviewing the sensitivity labels on files that are embedded in agents as knowledge, confirming the default sensitivity label policy and the usage rights that labels grant, and reviewing agents with embedded files in the Microsoft 365 admin center.
User Impact: Medium – Users who interact with agents may see sensitivity label enforcement on content that was previously unlabeled or differently labeled, and agents may restrict access to embedded files based on label policies.

Overview​

Agent Builder in Microsoft 365 Copilot lets makers upload files from their device as knowledge for an agent. The uploaded files become embedded content of the agent, and any user who can access the agent can get responses grounded in those files. If the original files carry sensitivity labels, those labels must keep protecting the content after it is embedded. Otherwise the agent becomes a way to expose sensitive content without the protections the label was meant to enforce.

Managing sensitivity labels for agent embedded content means understanding three things. First, a sensitivity label is applied to the embedded content automatically. It is whichever has the higher priority: the highest priority label on any embedded file, or the label from the organization's default sensitivity label policy. Second, that label controls who can use the agent: only users who have the extract usage right for the label can add and use the agent. Agent responses are labeled too, with whichever label has the higher priority among the embedded content and the agent's other knowledge sources. Third, administrators can filter the agent inventory in the Microsoft 365 admin center to find agents that use embedded files and see the sensitivity label of each file. This labeling applies only to agents built in Agent Builder with embedded files; content that Agent 365 instances create does not inherit sensitivity labels from its sources. Microsoft Purview Information Barriers are not supported on embedded files.

This activity supports Use Least Privilege Access by ensuring that sensitivity labels continue to enforce access boundaries even when content flows through agent interactions. A document labeled "Confidential - HR Only" should not become accessible to a broader audience simply because it was embedded in an agent. It also supports Verify Explicitly by confirming, for each agent with embedded files, which label applies and who has the rights to use it, rather than assuming that the original file's protection carries over.

Without this review, makers can embed unlabeled sensitive files, and the content of those files reaches every user who is given access to the agent. Threat actors who gain access to such an agent can retrieve sensitive content without triggering the access controls that a label would have enforced.

Reference​