Vocabulary
Glossary.
Words to argue with. Each term links to the research where we used it, so you can check what we meant by it.
The core terms
- Agent Experience (AX)
The experience AI agents have when discovering, choosing, and using your technology. Practicing AX means systematically improving that experience. Measurement is how you know whether you did.
Coined by Mathias Biilmann of Netlify in January 2025.
- Developer Agent Experience (DevAX)
Agent Experience for coding agents and the developers they work for: the experience a coding agent has when a developer asks it to build with your SDK, API, CLI, or framework. It’s what this site and its research focus on.
A developer asks for a new SPFx web part. Does the agent scaffold the current version, or one from 2020?
- Propensity
Whether the agent finds your technology and chooses it. Test it with tasks that don’t name your product.
Ask for “add email sending to this app” and see what the agent reaches for.
- Efficacy
Whether the agent uses your technology correctly once it has chosen it. Test it with tasks that name your product.
Does the agent call your current API, or the one you deprecated?
- AX stack
The layers that shape what an agent does: the model, which is fixed; the harness, which you influence through its vendor; and everything the agent reads and calls, which is yours.
How to improve AI coding agent support for your SDK, API, or CLI
- Lift
Against the bare baseline, a change that improves outcomes.
Structured JSON output from a CLI: 72 to 85 of 85 configuration checks.
- Drag
Against the bare baseline, a change that leaves outcomes the same or makes them worse.
Passing CLI arguments as one JSON payload: no gain in correctness, 4–11× the cost per task.
- Expensive lift
A change that improves outcomes at a cost too high to keep. Lift at 3× the tokens is a net loss.
One more point of quality at 5.6× the cost per run.
- Task cost
What it costs to complete one task with a given setup, counting every token at its price, not just the token count. Report it next to quality, every time.
A model with 33% cheaper tokens cost 3.7× more per run.
- Agent extensions
Everything you put in front of the agent to shape its behavior: skills, MCP servers, instruction files, and custom agents.
More terms
- Bare baseline
The same tasks run with no extensions, just the harness and the model. It’s what you compare every change against.
- Agent profile
The setup you ran with: operating system, harness, model and its settings, and the extensions loaded. A result belongs to the profile it was measured with.
- Harness
The agent app that runs the model. It controls the system prompt and how context is assembled.
- Scenario
A starting workspace, a realistic prompt, and the criteria the result is judged by.
- Readout
The report of an evaluation: gates, criteria, profile comparison, cost, and what to change next.
- Trajectory
The step-by-step record of an agent run: what it read, what it called, what it reasoned. ATIF is an open format for it.
- Composition tax
The value lost when extensions run together. An extension that produces lift on its own can produce drag next to another one.
- Training data gravity
Agents reach for the tool with the most training data, even when a newer one replaced it.
- Closest-match trap
When the model doesn’t know your SDK, it maps it to the nearest one it does know.
- Plan invalidation
Telling the agent that the approach it already planned will fail. A tip offers an option; a plan-breaking warning changes what the agent does.
A tip: 1 of 5 runs used the tool. A warning: 5 of 5.
- Skillmaxxing
Installing skills for everything. Their descriptions cost tokens before you type a word: 50 skills is about 10,000 tokens.
- Minimal viable model
The cheapest model that meets your requirements.