How to Learn from Negative Trajectories in Coding Agents

Failed Trajectories Aren’t Uniformly Bad

When training small language models to act as coding agents, the standard recipe is straightforward: collect successful trajectories from a larger teacher model, filter for the ones that reach the correct solution, and fine-tune on those. This pipeline discards a large fraction of the data: negative trajectories. Furthermore, scaling this pipeline to collect more data requires multiple inference attempts just to accumulate a sufficient positive pool.

On the other hand, training on failures naively can teach the model the wrong behaviors, and it's not obvious how to separate useful signal from noise in a trajectory that receives negative reward. So most work simply doesn't use negative trajectories at all.

However, a failed trajectory is not uniformly bad. It might contain a correct file localization step, a reasonable hypothesis about the bug, and then one wrong edit that breaks everything. The trajectory fails, but most of what happened before that wrong turn is still worth learning from. The question is how to make that explicit to the model during training.

Our approach is simple: for each negative trajectory, we ask another model to extract specific facts—the mistaken beliefs the agent acted on—and inject them into the user prompt as explicitly labeled wrong knowledge. When we recycle negative trajectories for SFT, we include the wrong facts in the context of that trajectory so that the model is aware of what it did wrong: here is what is wrong in the current trajectory. This is the "privileged information"—derived from the gold patch and available only at training time, not at inference. It primes the model with knowledge of what was wrongly inferred, so it can more easily learn from the rest.

How We Recycle Negative Trajectories

Our setup involves supervised fine-tuning a Qwen3-32B-Instruct model on teacher model (MiniMax M2.5)'s trajectories. To establish a baseline, we first train a 32B model solely on the teacher's positive trajectories from the SWE-Rebench dataset. Then, we train another 32B model on both positive and negative trajectories, where the negative trajectories are augmented with the privileged information about what was wrong. Our approach yields a 4.1% improvement, achieving 58.05 on SWE-bench Verified with under 4.5K trajectories on Qwen3-32B-Instruct model. Our approach also yields a 2.8% improvement on the Qwen3-4B-Instruct model with under 37K total trajectories.

How we utilize negative trajectories

Here is a real example of a labeled wrong fact:

Below is the structure of the privileged-information block added to user prompts in the training data. During the evaluation, we remove the <information> block.

Why Do Negative Trajectories Help?

Couldn't this just be that we train on more data?

To rule this out, we scaled the positive-only data from 3.7K to 15K trajectories. Pass@1 plateaued at the 3.7K baseline (pass@3 improved only marginally), so 11K additional positive trajectories yielded no measurable gain on pass@1. By contrast, adding just 0.7K negative trajectories to the 3.7K baseline lifted both pass@1 and pass@3 to the 15K positive trajectory level. A small amount of negative data matched what a large amount of extra positive data could not.

Could it be diversity—more unique tasks?

Since negative trajectories cover tasks the positive set didn't, the gains might come from added task diversity rather than the negatives themselves. But the matched-task comparison rules this out: adding 0.7K negative trajectories (→54.50) matches adding 1K positive ones (→53.95) on pass@1 and beats them on pass@3 (68.40 vs 67.6)—more gain from fewer examples. The effect is starker at scale: 0.7K negatives lift the 3.7K-positive base by 4.1% (→58.05), while growing the positive set from 3.7K to 15K adds just 0.15 (→54.1). So the two sources of signal aren't equivalent—failed trajectories carry information positives don't: what went wrong. If we simply train on raw negatives, this signal comes with a risk: training on failures can nudge the model toward the mistakes it should avoid. Labeling the wrong facts explicitly in the prompt (privileged information) mitigates this, letting the model learn what to avoid rather than imitating it.

Does Privileged Information Help?

To isolate the effect of privileged information (PI), we compare three training configurations: a baseline trained on positive trajectories only, a variant augmented with negative trajectories but without PI, and our full method that pairs negative trajectories with PI.

Adding negative trajectories without PI lifts Pass@1 from 53.95 to 56.67—the diversity and failure-signal benefit we saw earlier, captured even from raw negatives. Pairing them with PI reaches 58.05, isolating the additional contribution of labeling what went wrong. Marking the wrong facts in the user prompt lets the model treat the failed actions as conditioned on that flawed context rather than as behavior to imitate, so it internalizes the mistakes less even while still learning from negative trajectories. The full 4.1% gain over the positive-only baseline has non-overlapping error bars, so it isn't seed noise; the raw→PI step (56.67 → 58.05) is smaller and within error bars but points the same direction and is echoed on Pass@3 (69.6 → 70.4). Pass@3 rises consistently across all three settings (67.6 → 69.6 → 70.4), suggesting negative trajectories don't just raise average accuracy but broaden the set of problems the model can solve given a few attempts—a property we'd expect to give RL a richer reward signal to work with.

Are All Negative Trajectories Equally Helpful?

Unlike positive trajectories used in SFT training, negative trajectories inherently exhibit greater variance in quality. Some may have followed the right problem-solving strategy overall, only receiving a reward of zero due to a minor mistake at the very end. Others may have performed incorrect reasoning from the outset—stemming from buggy file localization—making the entire trajectory something the model should not learn from.

We approximate negative trajectory quality based on the ratio of wrong facts in the trajectory summary. A higher wrong ratio indicates a trajectory with little learning value. Negative trajectories with a wrong ratio of 80% or above are filtered out from the dataset.

What Is Happening in the Model with Negative Data?

Our analysis reveals a two-stage effect of negative data with PI. Our analysis was computed on a single run, hence small differences from the averages in the above tables. First, training on negative trajectories alone teaches the model exploration: it learns to run more tests, backtrack when stuck, and recover from failed attempts—behaviors seen less often in positive-only training. This increased persistence directly translates to higher solve rates, but at a cost: the model explores more broadly and takes longer to commit to a fix.

Adding privileged information then acts as a focusing lens: because the model has seen ground-truth diagnostics explaining why a trajectory failed, it learns to localize bugs faster—cutting the number of steps before its first edit, reducing aimless exploration, and suppressing redundant verification loops—all while preserving the solve-rate gains from negative data.

The table below breaks down this two-stage effect quantitatively across behavioral metrics. The first group captures overall performance; the second group (Effect 1: exploration) shows how negative trajectories increase thoroughness; the third group (Effect 2: focus) shows how PI then sharpens efficiency.

Transferability to a Different Model Size

To test whether the effect holds at a different scale, we repeated the experiment on Qwen3-4B-Instruct-2507. The base mixture is from BugPilot [Sonwane et al., 2025]—25K trajectories collected from two teacher models, Claude 4 [Anthropic, 2025] and MiniMax-M2.5 [MiniMax, 2025]. To keep compute manageable, we trained for a single epoch rather than the full three epochs of SFT training. On top of the base mixture we added positive or negative trajectories at two budgets—roughly 3K and roughly 11K examples—and compared the two datasets. We followed the same setup for the negative datasets: we filtered out trajectories with a wrong-fact ratio above 80% and added privileged information to the user prompt. We report the average Pass@1 resolution rate on SWE-bench Verified.

Two patterns from the main experiments reappear on the small model. First, adding positive data saturates: going from 3K to 11K positives leaves Pass@1 essentially unchanged (28.8 → 28.6, within error), consistent with the diversity ceiling seen earlier—once the added positives stop introducing new tasks, more of them contribute little. Adding negatives does not saturate performance: negative trajectories with PI improve the model from 29.2 at 2.6K to 30.4 at 12K. Second, at a comparable data budget, negative trajectories are at least as sample-efficient as positive ones: at the smaller scale, negative trajectories match positive-only performance while using fewer examples (29.2 vs. 28.8, within error; 2.6K vs. 3K), and at the larger scale, it yields a clear improvement (30.4 vs. 28.6, non-overlapping error bars). The effect holds on a smaller model on top of the base mixture built with enough positive trajectories.

Related Work

Code Agent Training

After SWE-agent [Yang et al., 2024] introduced the Agent-Computer Interface (ACI) for repository-level code editing, SWE-RL [Wei et al., 2025] trained coding agents using RL over real GitHub issue and PR histories. A wave of follow-up work—SWE-smith [Yang et al., 2025] and SWE-Fixer [Xie et al., 2025]—achieved strong results on SWE-bench by training on large-scale demonstration data via SFT. However, all of these approaches discard failed rollouts and train exclusively on successful trajectories.

Learning from Negative Trajectories

Wang et al. [2024] were among the first to show that failed trajectories carry useful signal: simply prefixing training examples with a success/failure label yields large gains on math reasoning and QA tasks. Song et al. [2024] proposed Trial and Error, building contrastive pairs from failure trajectories and training with DPO in an iterative loop. An et al. [2024] explore whether LLMs can learn from mistakes (LEMA), incorporating mistake-correction data pairs during fine-tuning. Taking this further, Lan et al. [2025], Peng et al. [2026] found that even expert failure trajectories contain helpful early steps—mistakes tend to occur only in the final few actions, meaning the bulk of a failed rollout is still informative and able to improve the policy of the models. Shrivastava et al. [2026] also found that failed rollouts contain rich evidence about how the environment responds, which agents can learn from.

Learning from Feedback

Beyond binary success/failure labels, a parallel line of work uses richer feedback as a training signal. RLTF [Song et al., 2026] treats text feedback as an intermediate signal between sparse scalar rewards and expensive full demonstrations—richer than a binary label, yet cheaper to collect than complete expert trajectories. IXT [Cui et al., 2025] annotates training data with natural language critiques via a thinking reward model, then prefix-conditions the model on this feedback, achieving up to 2.8× compute efficiency. RLCF [Viswanathan et al., 2025] extracts instruction-specific checklists and scores each criterion individually to compute rewards for RL, outperforming standard alignment baselines. π-Distill [Penaloza et al., 2026] trains a privileged information-conditioned teacher and an unconditioned student jointly within the same model, transferring training-time privileged information to a policy that acts without it. Adding annotations to SFT data has also been shown to preserve the pretraining distribution and mitigate semantic mode collapse during fine-tuning [Springer et al., 2025] .

Our work builds on these threads. Rather than labeling trajectories as simply pass or fail, we explicitly identify which knowledge within a failed coding trajectory was wrong and inject it as privileged information in the prompt during SFT, so that the wrong information attends to the prompt and is not internalized into the model's parameters.

Acknowledgements

Thanks to Emiliano Penaloza and Jeonghye Kim from Microsoft Research Montreal, and Akshay Krishnamurthy and Sadhika Malladi from Microsoft Research New York for helpful discussions and feedback.

Citation

@misc{negativePI,
  title  = {How to Learn from Negative Trajectories in Coding Agents},
  url    = {https://microsoft.github.io/debug-gym/blog/2026/08/negative-pi/},
  author = {Kim, Minseon and Caccia, Lucas and Moldavskaya, Darya and Pereira, Matheus and Shi, Zhengyan and Vera, Fabio and Singh, Chinmay and Côté, Marc-Alexandre and Yuan, Xingdi and Sordoni, Alessandro},
  month  = {August},
  year   = {2026}
}

References

1. An, S. et al. Learning From Mistakes Makes LLM Better Reasoner. 2024. arXiv:2310.20689
2. Song, Y. et al. Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents. 2024. arXiv:2403.02502
3. Wang, R. et al. Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents. 2024. arXiv:2402.11651
4. Yang, J. et al. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. 2024. arXiv:2405.15793
5. Cui, B. et al. Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages. 2025. arXiv:2605.20285
6. Lan, L. et al. Exploring Expert Failures Improves LLM Agent Tuning. 2025. arXiv:2504.13145
7. Sonwane, A. et al. BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills. 2025. arXiv:2510.19898
8. Springer, J. M. et al. Annotations Mitigate Post-Training Mode Collapse. 2025. arXiv:2605.09995
9. Viswanathan, V. et al. Checklists Are Better Than Reward Models For Aligning Language Models. 2025. arXiv:2507.18624
10. Wei, J. et al. SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution. 2025. arXiv:2502.18449
11. Wichers, N. et al. Inoculation Prompting: Instructing LLMs to Misbehave at Train-Time Improves Test-Time Alignment. 2025. arXiv:2510.05024
12. Xie, C. et al. SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution. 2025. arXiv:2501.05040
13. Yang, J. et al. SWE-smith: Scaling Data for Software Engineering Agents. 2025. arXiv:2504.21798
14. Zeng, A. et al. Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering Agents. 2025. arXiv:2504.09269
15. Penaloza, E. et al. Privileged Information Distillation for Language Models. 2026. arXiv:2602.04942
16. Peng, B. et al. Orchard: An Open-Source Agentic Modeling Framework. 2026. arXiv:2605.15040
17. Shrivastava, V. et al. ECHO: Terminal Agents Learn World Models for Free. 2026. arXiv:2605.24517
18. Song, Y. et al. Expanding the Capabilities of Reinforcement Learning via Text Feedback. 2026. arXiv:2602.02482
19. Anthropic. Claude 4 System Card. 2025. anthropic.com/news/claude-4
20. MiniMax. MiniMax-M2.5. 2025. github.com/MiniMax-AI/MiniMax-M2.5
21. Microsoft AI. MAI-Thinking-1: Building a Hill-Climbing Machine. 2026. microsoft.ai/pdf/mai-thinking-1.pdf