Prompt-level vs. agent-level
Chat Intelligence Report vs. Agent Intelligence Report
Both run on Purview audit data. Both ship as Power BI templates in the CopilotChatAnalytics repo. Chat Intelligence shows you what humans are doing with Copilot. Agent Intelligence shows you what your declarative and custom agents are doing. Most enterprises run both.
TL;DR
Run Chat Intelligence when the question is about user prompt patterns, depth, surface usage, and adoption-by-prompt-behaviour. Run Agent Intelligence when the question is about what agents are getting invoked, by whom, in which departments, and with what outcomes. They share PAX as the upstream data extractor, so if you've configured one, the second is a fast add.
| Chat Intelligence Report | Agent Intelligence Report | |
|---|---|---|
| Primary subject | Human Copilot prompts | Agent invocations |
| Repo | CopilotChatAnalytics | CopilotChatAnalytics |
| Format | Power BI .pbit | Power BI .pbit |
| License | Free · MIT | Free · MIT |
| Data source | Purview Audit (via PAX) | Purview Audit (via PAX) |
| Setup time | ~45 min once PAX is configured | ~30 min once PAX is configured |
| Prompts shown | Yes — prompt text, length, turn count | No (out of scope) |
| Surface breakdown | Word, Excel, Teams, Outlook, etc. | By agent surface |
| Agent invocations | Out of scope | Yes — declarative, custom engine, Studio |
| Agent author breakdown | No | Yes |
| Department breakdown | Via AD/Entra attribute join | Via AD/Entra attribute join |
| Outcome / success signal | Inferred from follow-up turns | Agent-emitted outcome where available |
| Companion to AI-in-One | Yes | Yes |
What Chat Intelligence answers
- What surfaces are people using Copilot in (Word, Excel, Teams, Outlook, Loop)?
- How deep are their prompts — single turns or multi-turn conversations?
- Which departments are prompting most, and at what depth?
- Is prompt behaviour maturing month over month?
What Agent Intelligence answers
- Which agents are getting invoked, and by whom?
- Which agents are shipped but unused — candidates for sunset?
- Which authors and departments are driving agent adoption?
- How are declarative agents performing vs. custom engine agents?
Why most teams run both
Chat tells you about the human in front of Copilot. Agent tells you about the orchestration layer your team built around them. Together they answer "is our Copilot investment being used by people and by the agents our team built on top of it" — which is the question leadership is actually asking.
Because both pull from PAX, you only pay the setup cost once. After PAX
is configured against Purview Audit, both .pbit files can refresh on
the same schedule.
Run Chat Intelligence if…
- The question is about people, not agents
- You need prompt-level visibility
- You want surface-level depth analytics
- You're building a Copilot maturity model
Run Agent Intelligence if…
- Your team has shipped declarative or custom agents
- You need to defend (or sunset) agent investments
- Leadership wants per-author / per-department breakdowns
- You're evaluating Copilot Studio agent ROI