Every DevAX article, from the Microsoft Developer Blog and Waldek’s blog. Each item links to the post where it was published.https://microsoft.github.io/agent-experience/https://blog.mastykarz.nl/automate-workflow-users-agents/https://blog.mastykarz.nl/automate-workflow-users-agents/You want to test whether an agent can deploy an app to your platform. But before you can check the agent's work, your eval harness has to click through a portal and an interactive sign-in. Whose problem is that?Sun, 18 Oct 2026 10:47:39 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/honest-evals-prove/https://blog.mastykarz.nl/honest-evals-prove/Your coding agent scored 100% on its eval, and every criterion passed. Then someone tried to build the code it generated.Sat, 17 Oct 2026 16:40:09 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/eval-judge-know-mean/https://blog.mastykarz.nl/eval-judge-know-mean/Does the solution follow best practices? Your LLM judge answers without hesitation and explains its verdict convincingly. So which best practices did it check?Sat, 17 Oct 2026 16:05:39 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/models-better-agent-tools-smaller/https://blog.mastykarz.nl/models-better-agent-tools-smaller/You wrote your agent skill to fix what the agent kept getting wrong. What happens to those instructions when a new model stops getting it wrong?Sat, 17 Oct 2026 15:39:45 GMTWaldek MastykarzOpinionhttps://developer.microsoft.com/blog/introducing-the-agent-experience-ax-practitioner-playbook/https://developer.microsoft.com/blog/introducing-the-agent-experience-ax-practitioner-playbook/AI coding agents now pick the SDK, the version, and the pattern for your developers. The AX Practitioner Playbook is our method for finding out whether they get your technology right, and for fixing the sources they rely on when they don't.Fri, 09 Oct 2026 09:44:25 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/what-is-agent-experience-ax/https://developer.microsoft.com/blog/what-is-agent-experience-ax/Your agent says it's done, and the code compiles, but did it pick your technology? Did it use it correctly? Learn what Agent Experience (AX) is, how to measure it, and why the obvious fix isn't always the right one.Tue, 06 Oct 2026 09:18:05 GMTWaldek MastykarzResearchhttps://blog.mastykarz.nl/eval-user-success-product-behavior/https://blog.mastykarz.nl/eval-user-success-product-behavior/Your eval checks whether the agent called the tool you built, and every run passes. What does that tell you about the developer who asked for help?Sun, 04 Oct 2026 14:54:12 GMTWaldek MastykarzOpinionhttps://developer.microsoft.com/blog/knowledge-cutoff-is-a-poor-proxy-for-model-capability/https://developer.microsoft.com/blog/knowledge-cutoff-is-a-poor-proxy-for-model-capability/A model can fail on features released before its knowledge cutoff, then succeed on ones released after it. We tested hundreds of product changes and found that the date tells you far less than the work does.Mon, 21 Sep 2026 07:51:46 GMTWaldek MastykarzResearchhttps://blog.mastykarz.nl/eval-score-half-result/https://blog.mastykarz.nl/eval-score-half-result/Your agent's score improved. Its inference bill increased 15 times. Before you celebrate, decide what the improvement is actually worth.Sun, 20 Sep 2026 14:48:07 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/agent-eval-more-than-result/https://blog.mastykarz.nl/agent-eval-more-than-result/Your agent completed the task and every grader passed. What happened before the score reached 100%?Sun, 20 Sep 2026 09:30:34 GMTWaldek MastykarzOpinionhttps://developer.microsoft.com/blog/your-ai-coding-agent-evaluation-is-only-as-good-as-its-sandbox/https://developer.microsoft.com/blog/your-ai-coding-agent-evaluation-is-only-as-good-as-its-sandbox/Your AI coding agent passed the eval. But did the model know the answer, or did it find it somewhere on your machine? A correct answer can still invalidate your measurement.Wed, 16 Sep 2026 09:09:51 GMTWaldek MastykarzResearchhttps://blog.mastykarz.nl/knowledge-cutoff-ceiling-snapshot/https://blog.mastykarz.nl/knowledge-cutoff-ceiling-snapshot/A model's knowledge cutoff looks precise. But how much does that date actually tell you about what it knows?Sun, 13 Sep 2026 14:40:58 GMTWaldek MastykarzOpinionhttps://developer.microsoft.com/blog/your-work-might-not-need-the-smartest-model/https://developer.microsoft.com/blog/your-work-might-not-need-the-smartest-model/The smartest model can cost five times more and deliver the same result, or even a worse one. See how evaluating your own work helps you get more value from your agent budget.Wed, 09 Sep 2026 09:14:04 GMTWaldek MastykarzResearchhttps://blog.mastykarz.nl/test-knowledge-skill/https://blog.mastykarz.nl/test-knowledge-skill/Your knowledge skill gives reasonable answers to the questions you've tried. What would you find in the parts of the source you didn't think to test?Wed, 09 Sep 2026 09:11:51 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/coding-agents-learning-mode/https://blog.mastykarz.nl/coding-agents-learning-mode/Coding agents can take us from intent to production in minutes. What happens to the understanding we used to build along the way?Fri, 04 Sep 2026 20:03:36 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/your-product-didnt-get-worse/https://blog.mastykarz.nl/your-product-didnt-get-worse/Customers still like your product and keep renewing. Meanwhile, their expectations may be changing in ways your usual signals won't reveal.Thu, 03 Sep 2026 15:04:42 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/your-coding-agent-extension-needs-continuous-evaluation/https://blog.mastykarz.nl/your-coding-agent-extension-needs-continuous-evaluation/Your extension passed its evals. Days later, its code is identical, but can you still trust the result?Thu, 03 Sep 2026 10:46:10 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/the-red-pen/https://blog.mastykarz.nl/the-red-pen/AI can keep an entire working session in context. Yet changing one sentence still requires directions that sound like a confused treasure map.Mon, 31 Aug 2026 12:31:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/your-app-is-losing-its-grip-on-your-customers/https://blog.mastykarz.nl/your-app-is-losing-its-grip-on-your-customers/Your customers still use your app and renew their contracts. Meanwhile, the work they once did in it might already be moving somewhere else.Sun, 30 Aug 2026 15:47:55 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/does-every-agent-skill-need-eval/https://blog.mastykarz.nl/does-every-agent-skill-need-eval/A skill can make an agent better and still be a bad trade. How much evidence should you expect before trusting someone else's skill?Fri, 28 Aug 2026 16:13:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/whats-volatile-about-your-technology/https://blog.mastykarz.nl/whats-volatile-about-your-technology/A model's knowledge cutoff doesn't tell you how old its knowledge of your SDK is. Technology providers know something more useful.Wed, 26 Aug 2026 14:03:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/agent-start-hypothesis-not-plan/https://blog.mastykarz.nl/agent-start-hypothesis-not-plan/Agents can turn a plausible first answer into a plan before current documentation gets a say. By then, correcting an outdated assumption is already expensive.Wed, 26 Aug 2026 13:30:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/dry-wrong-agent-documentation/https://blog.mastykarz.nl/dry-wrong-agent-documentation/DRY documentation saves maintainers from repeating themselves. What if it makes every agent pay for that convenience?Wed, 26 Aug 2026 09:00:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/your-agents-should-learn-for-the-organization/https://blog.mastykarz.nl/your-agents-should-learn-for-the-organization/Your agents discover stale guidance and hidden constraints every day. When the conversation ends, who gets to keep what they learned?Mon, 10 Aug 2026 11:00:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/every-team-needs-knowledge-interface/https://blog.mastykarz.nl/every-team-needs-knowledge-interface/Organizations have spent years improving how people find knowledge. Agents don't just find it. They make decisions with it.Sat, 08 Aug 2026 15:35:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/inference-efficiency-protecting-attention/https://blog.mastykarz.nl/inference-efficiency-protecting-attention/Every extra tool call looks like a small tax. The larger cost is what it takes away from the agent while it works on your problem.Mon, 03 Aug 2026 15:05:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/measuring-ai-productivity-wrong-way/https://blog.mastykarz.nl/measuring-ai-productivity-wrong-way/When we talk about AI and productivity, we ask how much time it saved us. That's the wrong question. The one that matters is how much time it saved everyone. AI didn't remove the cost of communication. It shifted it from the author to the audience, and we're celebrating the wrong side of the ledger.Thu, 30 Jul 2026 11:50:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/every-configuration-option-question-couldnt-answer/https://blog.mastykarz.nl/every-configuration-option-question-couldnt-answer/Open your favorite AI coding agent and you're greeted with a surprising number of decisions. Which model? How much reasoning? Which MCP servers? Most of us don't know the right answers. We treat that as the cost of cutting-edge AI. It's a sign the technology hasn't matured yet.Wed, 29 Jul 2026 11:30:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/next-literacy-externalizing-judgment/https://blog.mastykarz.nl/next-literacy-externalizing-judgment/When ChatGPT took off, everyone wanted to learn prompt engineering. We've been optimizing the wrong thing. The biggest productivity gains from LLMs come from learning to externalize judgment.Tue, 28 Jul 2026 13:54:00 GMTWaldek MastykarzOpinionhttps://blog.mastykarz.nl/what-is-task-worth-ai-agents/https://blog.mastykarz.nl/what-is-task-worth-ai-agents/We keep asking whether AI coding agents are good or bad, fast or slow, cheap or expensive. But good compared to what? Until you know what a task is actually worth to you, you have no way to tell whether the agent's output was a bargain or a rip-off.Fri, 24 Jul 2026 10:21:00 GMTWaldek MastykarzOpinionhttps://developer.microsoft.com/blog/how-to-test-agent-experience-changes-without-shipping-them/https://developer.microsoft.com/blog/how-to-test-agent-experience-changes-without-shipping-them/Most changes you think will improve AI agent behavior won't. We tested a dozen hypotheses on a real project upgrade scenario and the majority failed. Learn how to emulate documentation, API, and MCP server changes locally so you can validate what works before shipping anything to production.Tue, 21 Jul 2026 07:15:12 GMTWaldek Mastykarz, Garry TrinderResearchhttps://developer.microsoft.com/blog/how-to-test-agent-skills-without-hitting-real-apis/https://developer.microsoft.com/blog/how-to-test-agent-skills-without-hitting-real-apis/Your agent skill calls an API. The moment you start evaluating it, every run either costs money or mutates production data. Learn how to mock APIs transparently so you can run evals without changing your skill or hitting real endpoints.Fri, 17 Jul 2026 09:27:50 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/building-ax-evals-that-actually-work/https://developer.microsoft.com/blog/building-ax-evals-that-actually-work/Wed, 15 Jul 2026 12:53:13 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/the-hidden-variables-in-your-agent-eval/https://developer.microsoft.com/blog/the-hidden-variables-in-your-agent-eval/Wed, 08 Jul 2026 12:11:17 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/dont-rewrite-your-cli-for-agents/https://developer.microsoft.com/blog/dont-rewrite-your-cli-for-agents/There’s advice making the rounds: replace your CLI args with a single --json payload so agents can use your tool more effectively. The thinking being, that agents already think in structured formats, and nested data maps cleanly to JSON.Tue, 07 Jul 2026 13:52:06 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/not-all-model-upgrades-are-upgrades/https://developer.microsoft.com/blog/not-all-model-upgrades-are-upgrades/A new model drops with lower per-token pricing and better benchmarks. You switch. A week later someone asks why the agent is burning 12x more tokens on the same task while producing worse output.Mon, 06 Jul 2026 07:49:44 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/what-ai-benchmarks-are-not-telling-you/https://developer.microsoft.com/blog/what-ai-benchmarks-are-not-telling-you/Wed, 01 Jul 2026 14:31:55 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/your-agent-already-has-a-plan/https://developer.microsoft.com/blog/your-agent-already-has-a-plan/If an agent isn’t doing the right thing, the obvious move is to make the docs clearer. Add a tip, spell out the correct command, describe the right approach more prominently. You do all of that, and the agent still ignores it.Fri, 26 Jun 2026 00:00:05 GMTGarry TrinderResearchhttps://developer.microsoft.com/blog/when-the-model-has-never-seen-your-code/https://developer.microsoft.com/blog/when-the-model-has-never-seen-your-code/Wed, 24 Jun 2026 14:04:52 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/models-dont-have-preferences-they-have-context/https://developer.microsoft.com/blog/models-dont-have-preferences-they-have-context/You open a fresh chat, type “What framework should I use for a web app?”, and the model says “React.” You screenshot it, share it, and write “Claude prefers React.” It gets engagement. People nod along. A few reply with their own results.Mon, 22 Jun 2026 14:53:23 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/stop-overloading-your-skills/https://developer.microsoft.com/blog/stop-overloading-your-skills/You built a skill for your technology. API references, authentication flows, SDK patterns, error handling, version info, all packed into one skill. The agent calls it, gets all that context, and generates code.Thu, 18 Jun 2026 14:25:27 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/when-your-agent-extensions-fight-each-other/https://developer.microsoft.com/blog/when-your-agent-extensions-fight-each-other/Your extension works in isolation. You measured it, confirmed it creates lift. Then a developer installs it alongside 14 other extensions, and outcomes get worse. The symptoms look like a bug in your code, but the problem is elsewhere: extensions fighting each other for the same context window and model attention.Wed, 17 Jun 2026 14:55:53 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/competing-against-yourself/https://developer.microsoft.com/blog/competing-against-yourself/You deprecated the old CLI and shipped something better. Developers are migrating, but AI coding agents aren’t. They keep reaching for the deprecated tool, confidently scaffolding projects with something you sunset months ago.Tue, 16 Jun 2026 14:55:23 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/your-agent-just-scaffolded-a-project-from-2020/https://developer.microsoft.com/blog/your-agent-just-scaffolded-a-project-from-2020/Your agent ran a scaffold command. Project generated, dependencies resolved, no errors. Everything looks fine. Except it’s based on the project structure from 2020, and neither you nor the agent noticed.Thu, 11 Jun 2026 17:54:29 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/is-your-agent-extension-actually-working/https://developer.microsoft.com/blog/is-your-agent-extension-actually-working/You shipped your skill. Everything looks like it's working. But is the generated code actually better because of your extension? You can't tell without measuring, and measuring agent extension impact is harder than it looks.Wed, 10 Jun 2026 19:06:17 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/stop-skillmaxxing-save-your-tokens/https://developer.microsoft.com/blog/stop-skillmaxxing-save-your-tokens/You built a dozen skills for your technology: authentication, CRUD, error handling, deployment, testing, monitoring. Then you installed a cloud platform bundle with 15 more covering diagnostics, storage, compliance, and cost optimization.Wed, 10 Jun 2026 18:00:15 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/how-ai-coding-agents-actually-use-your-technology/https://developer.microsoft.com/blog/how-ai-coding-agents-actually-use-your-technology/You ship an SDK, a CLI, an API, and developers use it. Now AI coding agents use it too, except they use it differently than humans do.Wed, 27 May 2026 18:30:38 GMTWaldek MastykarzResearchhttps://developer.microsoft.com/blog/the-ax-stack-whats-fixed-where-you-can-win/https://developer.microsoft.com/blog/the-ax-stack-whats-fixed-where-you-can-win/AI coding agents use your SDK, API, or CLI, but often get it wrong. The stack has layers you can't change and one you can: agent extensions. Learn how to measure whether yours are helping or hurting.Thu, 21 May 2026 17:39:59 GMTWaldek MastykarzResearch