Data Query
This category evaluates an agent harness and model (or MCP Host) on its ability to retrieve data
from a live Business Central environment to answer a natural-language data question. It is
execution-based (no LLM judge): the agent reports the rows it retrieved (answer.json), and the
run is resolved when those rows match the result set of a hidden gold AL query run against the same
Cronus/Contoso demo data (values compared normalized; order ignored unless the entry is ordered).
The point of the category is to compare how the data is retrieved:
- Baseline — no data tooling. The agent has to reach the answer on its own (e.g. authoring an AL query from knowledge of the schema), which is hard on a low-resource domain language.
- BC MCP experiment — the agent is given Business Central’s Data Query MCP tools
(
bc_data_find_tables,bc_data_get_table_schema,bc_data_get_table_relations,bc_data_query) so it can discover tables, inspect schemas and relations, and compile/run read-only AL queries against the live environment. The agent is isolated so the MCP endpoint is its only route to the data, which keeps the comparison honest.
Baseline Leaderboard
No results available yet. Check back soon!
BC MCP Experiment
Comparing runs that enable the Business Central Data Query MCP tools (bc-mcp) against the
matching no-tooling Default baseline for the same model.
No results available yet. Check back soon!