Establish Agent Publishing and Certification Standards
Implementation Effort: Medium – Requires cross-team alignment between security, platform engineering, and development teams to define publishing standards, certification criteria, and observability instrumentation requirements.
User Impact: Low – Standards apply to agent developers and publishers; end users are not directly affected.
Overview
Organizations building or integrating agents need clear standards that define how agents are registered, instrumented, certified, and made available in the enterprise Agent Registry. Without these standards, the registry becomes cluttered with incomplete metadata, inconsistent naming, and agents that lack proper ownership attribution or runtime telemetry. This directly undermines the Zero Trust principle of Verify Explicitly because administrators cannot make informed access decisions about agents they cannot properly identify, and security teams cannot verify agent behavior without standardized observability data.
Publishing standards should cover naming conventions, required metadata fields (description, owner, data access scope, intended audience), and the process for submitting agents for review before activation. For third-party agents built on non-Microsoft LLMs, the Custom Engine Agent framework provides the integration path into the Microsoft 365 ecosystem — but the framework alone does not enforce governance. Standards must require that every agent published through this framework meets the same metadata and review requirements as first-party agents built in Copilot Studio.
Certification standards establish the evidence-based process for verifying that an agent meets security, compliance, and operational requirements before production deployment. A key component is requiring integration of the Agent 365 Observability SDK, which provides standardized telemetry instrumentation that agents use to emit structured traces, metrics, and events about their runtime behavior — what tools were called, what data sources were accessed, how long operations took, and what outcomes were produced. This telemetry populates the Agent 365 admin console views and enables certification reviewers to verify that an agent calls required safety APIs (such as Prompt Shields and groundedness detection), respects rate limits, accesses only declared data sources, and handles errors appropriately. Without this telemetry, certification relies on code review and documentation alone, which cannot capture runtime behavior.
This task supports Use Least Privilege Access by requiring developers to declare the minimum permissions and data access scopes their agent needs at publish time, rather than requesting broad access after deployment. It supports Assume Breach by ensuring that when an agent behaves anomalously — accessing unexpected data sources, calling tools outside its declared scope, or generating unusual error patterns — the telemetry from the Observability SDK is available for security teams to investigate. Organizations that do not establish publishing and certification standards allow agents to enter production without consistent governance, leaving security teams without the context or telemetry needed to evaluate whether an agent's behavior is legitimate.