Frequently asked questions

Common questions about the Agent Review Program and Agent Review Center, and the error messages people run into most often when they set it up, with how to fix them.

Select a question to see the answer. To find an error, select Expand all, then search the page (Ctrl+F) for the error code or the exact wording on your screen. Setting up for the first time? Start with the installation guide.

About the program

What is the difference between the Agent Review Program and Agent Review Center?

The Agent Review Program is the expert-guided service: Microsoft experts work with you to review your agent, walk through the findings, and validate your fixes. Agent Review Center is the tool behind it. It runs in your own environment, and it stays there after the engagement so you can review your agents again whenever you want.

See The program and the tool.

Is my agent a good fit for the program, even with little traffic?

The program suits an agent in testing, pilot, or production that has a meaningful user base and enough real conversations for the findings to tell you something.

An agent with little or no traffic can still have its configuration and orchestration reviewed, but the runtime analysis will have little to work with. Turn on transcript recording now, so conversations build up before the review.

Not sure? Ask your Microsoft contact, who scopes it with you before Session 1.

How long does the program take, and how much of my time does it need?

Three sessions across four weeks: a 1-hour session in week 1, then 1–2 hour sessions in weeks 2 and 4. You make the agreed fixes in week 3. Plan for one named owner with about 3–6 hours across the sessions. See How the program runs.

How do I join the program?

Use the nomination options on the overview page, or ask the Microsoft team you already work with about the Agent Review Program. Your Microsoft contact scopes it with you before anything starts.

How do I contact the Agent Review Program team?

Email the team at serenaxie@microsoft.com. Don’t include transcripts or other sensitive data. During an engagement, you can also go through your Microsoft contact.

Who should join Session 1, and how do I prepare?

Who: the agent’s owner, the person who will work on the agent. Session 1 includes setting up Agent Review Center, so also have someone available who can change environment settings and assign roles: a Power Platform admin, an environment admin, or a System Administrator.

Before the session:

  • Turn on transcript recording in the agent’s environment as early as you can. It is not retroactive.
  • Ask your admin to work through steps 1 to 3 of the installation guide: environment settings, and data policies that allow the three connectors.
  • Be ready to talk about the agent’s use case, target users, and deployment stage, what success looks like, and your top concerns.
What do I keep after the program?

Agent Review Center stays installed in your environment, with your reviews and their snapshots, so you can review your agents again whenever you need to. You can also download any result as a PDF or JSON report.

About Agent Review Center

What can Agent Review Center review?

Copilot Studio agents in your environment, and declarative agents and skills uploaded as files (.zip, .md, .yaml, .yml, or .json).

Runtime analysis from conversation transcripts is available for Copilot Studio agents in the environment where Agent Review Center runs. Declarative agents and skills are reviewed from their files, so live connected agents can’t be looked up.

What’s in Agent Review Center?
  • Scorecard: a grade and a score rolled up across the agents you review, with a tab for each agent.
  • Findings: filter by component and status. Mark a rule as a manual pass when it doesn’t apply to your agent.
  • Architecture map: how instructions, skills, tools, and knowledge connect, with a keyboard-accessible list view.
  • Runtime views: Errors, Latency, and Knowledge, from your own conversation transcripts.
  • Agent Debugger: step through one conversation: the steps the agent took, tool calls, a performance timeline, and estimated Copilot Credits.
  • Snapshots and progress: every run is kept unchanged, so you can compare results over time.
  • Share results: download a result as a PDF or JSON report.
  • Readiness checklist: checks that the environment is set up, and Copy admin link sends your admin straight to the setting to change.
How is the score worked out?

Each review gives a letter grade and a score out of 100. When a review covers several agents, the rolled-up score combines them, and each agent has its own tab with its own score.

The review summary shows the score and weight of each area that was checked. If the AI-assisted checks don’t run, their weight is spread across the rest and the score is marked as partial.

Can I trust the AI-generated findings?

Findings labeled AI generated come from the AI-assisted checks, and AI-generated content may be incorrect. Each finding shows its evidence and references so you can check it before you act.

If a rule doesn’t apply to your agent, select Mark as manual pass from the menu beside it. Restore the review’s verdict undoes it.

Is Agent Review Center the same as the Agent Review Tool in the Copilot Agent Kit?

No. The Copilot Agent Kit includes an earlier tool called the Agent Review Tool. Agent Review Center is distributed separately, as its own managed solution. If the kit is already installed in the environment you plan to use, check with your Microsoft contact before you import Agent Review Center.

Where is my data stored?

Agent Review Center is installed in, and runs in, a Power Platform environment in your own tenant, against your own agents. Reviews and results are stored in Dataverse tables in that environment. Results are yours: sharing them outside your team is your decision.

For detailed data-handling questions, ask your Microsoft contact.

Does it collect telemetry?

The solution includes a cloud flow, Global | LogFeatureTelemetry I ARC, that the app calls when you use a feature, for example creating or re-running a review. The flow contains no connectors. For detailed data-handling questions, ask your Microsoft contact.

What happens if my files contain a secret?

If something in an uploaded file looks like a credential, it is withheld from the AI-assisted checks and reported as a finding. Remove the secret from the files, and review again.

Does running a review use Copilot Credits?

Yes. Running a review consumes Copilot Credits, which must be allocated to the environment where Agent Review Center runs. Each user also needs the licenses listed in What you need to bring.

Installing Agent Review Center

Who needs to be involved in setup?

A Power Platform admin for licensing, credits, data policies, and environment settings; a System Administrator in the environment to import the solution; the agent owner, to share the agent; and the reviewers themselves. See who’s who and verify the installation in the installation guide.

Which connectors must our data policies allow?

Three, together in the same group: Microsoft Dataverse, Agents, and Power Automate Management. The setup app uses Power Automate Management, and the import fails if it is blocked.

If your environment uses advanced connector policies, add all three to the allowed list. See Data policies and connectors in the installation guide.

Error Import fails with AcpDlpPolicyEvaluation: the shared_flowmanagement connector is blocked

What you see

The solution import stops with an error like this one. Your message shows your own policy name, IDs, and timestamps.

Solution "Agent Review Center" failed to import: CanvasApp import: FAILURE: Code: AcpDlpPolicyEvaluation Message: The app 'cat_copilotagentkitinstaller_81f96' cannot be saved because one or more connectors it uses 'shared_flowmanagement' are blocked by Advanced Connector Policy (ACP) or Data Policies (DLP) for this environment. Please contact your environment administrator '' to update the policies '<policy name>' to allow the connectors used by this app. Link for the policy wiki ''. Violation details: [PolicyId: <policy ID>, PolicyDisplayName: <policy name>, PolicyType: DLP, PolicySource: , PolicyLastModified: <timestamp>, ViolationType: BlockedConnector, Parameters: [shared_flowmanagement]]. Client Request Id: <request ID>.

How to fix it

  1. In the Power Platform admin center, go to Security → Data and privacy → Data policies. On older layouts, it's Policies → Data policies.

  2. Open the policy named in the error message (the PolicyDisplayName value).

  3. On the Prebuilt connectors step, find Power Automate Management and move it from Blocked to Business. It needs to be in the same group as Microsoft Dataverse and Agents (preview).

  4. Check that those two are also in Business. Otherwise, the next attempt will fail with a BusinessAndNonBusinessConnector violation instead.

  5. Save the policy, wait a few minutes for it to take effect, then retry the import.

Learn more: Data policies · Data policies and connectors in the installation guide

Error Import fails with AcpDlpPolicyEvaluation: an advanced connector policy blocks a connector

What you see

The import stops with an AcpDlpPolicyEvaluation error, like the one in the previous question. This time the violation names an Advanced Connector Policy (ACP), and the blocked connector can be a different one, such as shared_agentnode, the Agents connector.

Why it happens

Advanced connector policies block every connector that isn’t explicitly allowed, including new ones.

How to fix it

  1. Find where the policy is set. An environment has at most one advanced connector policy: set on the environment itself, or inherited from its environment group.

  2. For a single environment: in the Power Platform admin center, select Security > Data and privacy > Advanced connector policies. For an environment group: select Manage > Environment groups, open the group, and select Rules > Advanced connector policies.

  3. Add all three connectors the solution needs: Microsoft Dataverse, Agents, and Power Automate Management. Then select Save. For an environment group, also select Publish rules.

  4. Wait a few minutes, then retry the import.

Learn more: Advanced connector policies

I can’t edit the data policy named in the error

Environment admins can’t edit tenant-level policies. Ask a Power Platform admin to make the change, and send them Data policies and connectors from the installation guide.

Learn more: Manage data policies

The import fails, or the apps are missing after the import

The environment is not ready for code apps, or the import did not complete.

Confirm that Enable code apps is on in the environment, and that you are a System Administrator there. See Configure the environments in the installation guide. Also check that your data policies allow the three connectors.

I have a folder instead of a .zip file

Some browsers unzip downloads automatically. Import the .zip as it is: download it again with the download button in the installation guide, with automatic unzipping turned off.

Error The setup app says Setup could not complete

The setup app (Copilot Agent Kit Installer) could not confirm the connections or start the Review Agent Components flow.

Check that both connections exist and are signed in, and that your data policies allow Microsoft Dataverse, Agents, and Power Automate Management to be used together. Then open the setup app again. It repairs what it can on each run.

If it still can’t complete, follow Run the one-time setup app in the installation guide, and let your Microsoft contact know.

Error …connection references still need a connection.

A connection reference in the solution has no connection.

Run the setup app (Copilot Agent Kit Installer), or select the connection in the readiness checklist. To check by hand, see Run the one-time setup app in the installation guide.

Error …required flows are off. Map every connection first.

The Review Agent Components flow is off because a connection is missing.

Fix the connections first, then turn the flow on from the readiness checklist, or run the setup app again.

Error You need owner or co-owner rights on this flow to turn this flow on.

Your account cannot change the flow's state. The user who imported the solution owns its flows and connection references.

Ask the person who installed the solution, or a System Administrator, to turn it on in Power Automate.

Transcripts and runtime analysis

Error Conversation transcript recording is turned off for this environment

There are no transcripts to analyze.

Turn on transcript recording in the agent environment. See Configure the environments in the installation guide. Recording is not retroactive: only conversations after you turn it on are saved.

I just turned on transcripts. How far back can a review look?

Only to when recording was turned on. Transcript recording is not retroactive, so only conversations after it was turned on are recorded. This is why you turn it on before the review period starts.

Transcripts are also kept for 30 days by default, unless an admin changes the retention period. Run your first review while the conversations you care about are still there.

Learn more: Download conversation transcripts in Power Apps

Error No transcript records in this range

Nothing was recorded in the dates you chose.

Widen the date range. Recording only captures conversations after it was turned on.

Error Transcript viewing is blocked for this environment.

The analysis still runs, but individual conversations cannot be opened to confirm a finding.

Turn on Allow agent owners and editors to see session transcripts… for the environment. See Configure the environments in the installation guide.

Error Couldn't read conversation transcripts just now.

Transcripts could not be read at that moment.

Try again, or run the review without runtime analysis. Check that the reviewer has the Bot Transcript Viewer role. See Share with reviewers in the installation guide.

Can Agent Review Center review agents in another environment?

Configuration reviews can cover agents in other environments, but conversation transcripts are only analyzed in the environment Agent Review Center runs in. To include runtime analysis, install it in the same environment as the agent.

If you see Conversation transcripts are analysed only in the environment this app runs in., this is why. The configuration review still runs.

Running reviews and reading results

Error No active agents in this environment.

There is nothing to review in the environment you selected.

Select the environment where the agent lives, and check that the agent exists and is active there.

Error That file is larger than 16 MB. Upload a smaller artifact.

Uploads for declarative agent and skill reviews can be up to 16 MB. Remove files the review doesn’t need, such as large binaries, and upload again. Accepted file types are .zip, .md, .yaml, .yml, and .json.

Error The AI stage did not run for this review, and the score is partial

The AI-assisted checks did not run, so their weight was spread across the rest of the review.

Check connections, flows, and credits in the readiness checklist, then select Review again.

How do I share results?

Select Share results, then Download PDF report or Download JSON report. The menu tells you whether the snapshot includes runtime analysis or design findings only. The reports are yours: sharing them outside your team is your decision.

How do I show that my fixes worked?

Select Review again after you make your fixes. Each run is kept as a snapshot, and View progress shows the score across all of them. For runtime analysis, choose a date range that covers conversations after your fix.

Updates and removal

How do I know when there is a new version?

New versions are published as releases on GitHub and listed in the changelog. To be notified, select Watch > Custom > Releases on the repository.

How do I upgrade?

Import the new solution file over the existing one, and keep the default upgrade option. Your reviews and results stay in place. See Upgrade to a new version.

How do I remove it?

Delete the Agent Review Center solution in Power Apps. This also deletes every review and result, so download the reports you want to keep first. See Uninstall.

Installing for the first time? Follow the installation guide. During an engagement, raise anything not covered here with your Microsoft contact, or contact the Agent Review Program team.