← Orchard-Agentic
SECUREVIBE

SecureVibe: Making Vibe Coding
More Secure

Teaching coding agents to plan, code, and test for the security requirements
that users don’t explicitly ask for.

Danqing Wang1,2*, Baolin Peng2, Zhepei Wei2,3*, Isadora White2,4*,
Wenlin Yao2, Hao Cheng2, Qianhui Wu2, Minseon Kim2,
Xingdi Yuan2, Lei Li1, Jianfeng Gao2

1Carnegie Mellon University2Microsoft Research3University of Virginia, Charlottesville4University of California, San Diego

Carnegie Mellon UniversityMicrosoft Research

* Work done during an internship at Microsoft Research.

Paper Code Dataset BibTeX
THE USER’S REQUEST

“Implement template substitution for strings like /user/${user.id}, including nested properties.”

Vibe coding✓ Functional✗ Security
function render(tpl, ctx) {
  return tpl.replace(/\$\{([^}]+)\}/g,
    (_, expr) => eval("ctx." + expr));
}
CWE-94 · Code injection. Any text inside ${…} runs as code.
SecureVibe✓ Functional✓ Security
function render(tpl, ctx) {
  return tpl.replace(/\$\{([\w.]+)\}/g,
    (_, key) => lookup(ctx, key.split(".")));
}
Data, not code. Only dotted keys are resolved; nothing is evaluated.

ATTACK INPUT /user/${id; process.exit()} left: executesright: stays literal text

+6.9 ptsBaxBench security19.7% → 26.6%SecureVibe-rl
+11.5 ptsUnseen CWEs · SusVibes7.7% → 19.2%SecureVibe-hg
+4.1 ptsSWE-bench Verified60.9% → 65.0%SecureVibe-base

Pass@1 gain over the Qwen baseline, in percentage points.

TL;DR

SecureVibe teaches coding agents the security requirements users don’t state: SFT on a 1,648-example Security Suite (planning, coding, testing), then GRPO on execution rewards or hint-guided self-distillation. Security pass@1 rises on BaxBench and SusVibes, including unseen CWEs, and SWE-bench Verified improves too.

THE CHALLENGE

Functionally correct ≠ secure

Agents can pass every functional test and still ship a vulnerability, because users ask for a working feature, not a threat model.

TRY IT YOURSELF

A fix can work and still be unsafe

All three policies serve normal filenames. Only one keeps every file inside the data root.

Would this filename escape the root?

Pick an example or type a filename, then select Check path.

Illustrative lexical simulation; not a model rollout or filesystem test.

Ready to check. Select Check path to see the results.

THE RECIPE

Learn the behaviors. Then reinforce them.

SecureVibe training overview: Security Suite supervised fine-tuning followed by outcome-based GRPO or hint-guided on-policy self-distillation.
Qwen baselineQwen3.5-35B-A3B, before SecureVibe training
SecureVibe-base+ Security Suite SFT
SecureVibe-rlbase + GRPO (outcome rewards)
SecureVibe-hgbase + hint-guided OPSD

1. Build a foundation with the Security Suite

1,648 training examples across four tasks. Select one to replay a recorded example.

FOLLOW THE EVIDENCE

Loading case studies…

2. Improve with outcomes or hints

SecureVibe-rl · GRPO

Learn from execution

GRPO with rewards for a valid patch, functional tests, and security tests.

GRPO training guide
SecureVibe-hg · OPSD

Learn from a hinted teacher

On-policy self-distillation from a teacher that sees security hints. No hints at inference.

OPSD training guide

INSIDE SECUREVIBE-HG

Same task. Extra guidance for the teacher only.

STUDENT + TEACHER SEE

Generate a CSV report from ledger entries using a client-supplied filter and sort order.

ONLY THE TEACHER ALSO SEES

Treat filter values as data: bind them as SQL parameters. Choose column names and sort directions from an allowlist rather than inserting arbitrary user text into the query.

GENERALIZATION

Better security. Broader coding capability.

Qwen3.5-35B-A3B baseline vs. the three SecureVibe variants, all run with mini-swe-agent.

Evaluation
Metric

SecureVibe-hg improves SusVibes unseen-CWE subset security pass@1 from 7.69% to 19.23% (+11.54 points).

Functional: passes functional tests. Security: passes functional and security tests. SWE-bench Verified has no security metric.

BEHIND THE GAINS

What changes in the agent’s behavior?

Security is a process, not just an output.

The Security Suite produces more of all three security behaviors than either alternative SFT recipe.

Share of trajectories showing each behavior, by SFT recipe.

Hints help when successful outcomes are scarce.

SecureVibe-hg raises all three behaviors on both benchmarks; SecureVibe-rl’s largest gain is planning on AutoBax.

Behavior frequency (%), not pass rate.

BUILD ON SECUREVIBE

From training to evaluation

TRAINING

SecureVibe

SFT, reinforcement learning, and hint-guided distillation recipes.

 EVALUATION

SecureVibeEval

Benchmark execution, agent harnesses, and functional and security grading.

 DATA

Datasets

Training inputs and benchmark resources with source mappings and checksums.

Citation

@misc{wang2026securevibe,
  title={SecureVibe: Making Vibe Coding More Secure},
  author={Danqing Wang and Baolin Peng and Zhepei Wei and
          Isadora White and Wenlin Yao and Hao Cheng and
          Qianhui Wu and Minseon Kim and Xingdi Yuan and
          Lei Li and Jianfeng Gao},
  year={2026},
  eprint={2609.38606},
  archivePrefix={arXiv},
  primaryClass={cs.CR},
  url={https://arxiv.org/abs/2609.38606}
}