Cognition has launched Devin Security Swarm, an agentic system that scans large codebases for exploitable vulnerabilities, proves each one in an isolated sandbox, and opens a remediation pull request. On a benchmark of 50 real-world vulnerabilities, each tied to a published GitHub Security Advisory across 14 programming languages, it found 72 percent of them at 90.23 dollars per run, more than any competing AI scanner tested and roughly 30 percent cheaper per finding than the next most accurate tool. The headline is not the recall number on its own but what it signals: security scanning is shifting from pattern-matching lists of maybe-issues to agents that confirm exploitability before they file anything.
Background
Traditional static analysis reads code one file at a time and flags patterns that look risky. That approach produces long lists dominated by false positives, and it structurally cannot see business-logic flaws, chained authentication bypasses, or exploit paths that cross service boundaries, because those live in the relationships between files rather than in any single one. As AI-assisted development pushes more code into repositories faster, the gap between how quickly code is written and how quickly it can be reviewed keeps widening, and a scanner that mostly generates noise makes that gap worse rather than better.
Devin Security Swarm is Cognition's answer, and it extends the same trajectory covered in our look at Devin and AI coding-model routing: turning Devin from a single-task assistant into a full engineering agent. Here the job is defense, and the design leans on parallelism and runtime validation rather than a bigger rule set.
Deep Analysis
How the Swarm Actually Works
Security Swarm runs a swarm of parallel agents where each one investigates a segment of the codebase and reasons across files rather than in isolation. That cross-file reasoning is what lets it catch business-logic flaws, chained auth bypasses, and cross-service exploit paths that single-file analysis misses. Devin then composes individual findings into full attack paths and reproduces each one in an isolated sandbox to confirm exploitability at runtime. The parallel structure is described in Cognition's Agentic MapReduce writeup, which maps agents over codebase segments and then reduces their findings into validated attack chains.
Validation Is the Whole Point
The differentiator is not detection, it is confirmation. A scanner that only flags patterns hands a security team a queue of possibilities to triage. Security Swarm instead builds the attack path and runs it in a sandbox, so a finding arrives already proven exploitable, and only then does it write the patch and open a pull request for human review. In Cognition's evaluation, this let it surface three critical issues that every other tool missed: a PHP sandbox bypass via template injection, an argument injection through metadata value parsing, and an overly broad deserialization surface in Spring Kafka. Those are exactly the class of flaws that never show up as a simple pattern match.
The Benchmark, Read Honestly
The evaluation used 50 real vulnerabilities, each anchored to a published GitHub Security Advisory, spanning Go, Python, JavaScript, Rust, Ruby, C#, Java, Swift, PHP, Elixir, Erlang, C, Kotlin, and Dart. The results, cost included, tell a clearer story than recall alone.
| Tool | Recall | Cost per run |
|---|---|---|
| Devin Security | 72% | $90.23 |
| Claude Security | 68% | $131.87 |
| Codex Security | 48% | $118.20 |
| Cursor Security | 26% | $4.60 |
Devin leads on recall and is cheaper per run than the next two most accurate tools. Cursor is far cheaper but catches roughly a third as many issues, which is the tradeoff any team has to weigh: a cheap scan that misses most exploitable bugs is not obviously a bargain when a single missed vulnerability can cost far more than the scan.
Impact on Creators
If you ship AI-generated code, the review bottleneck is real and Security Swarm is aimed squarely at it. The practical value is a shorter, higher-signal queue: validated exploits with an attached fix PR instead of a wall of pattern alerts you have to sort by hand. It runs on customizable schedules, with the first full scan establishing a baseline and subsequent scans processing only the code that changed, which keeps recurring cost tied to churn rather than repo size. For a small team without a dedicated security engineer, that changes what is realistic. The right way to judge it is on the quality of its confirmed findings and the patches it proposes, not on raw finding count, because the entire premise is that a shorter list of proven issues beats a long list of possibilities.
Key Takeaways
- Devin Security Swarm found 72 percent of 50 real GitHub Security Advisory vulnerabilities at 90.23 dollars per run, leading the field on recall.
- It is roughly 30 percent cheaper per finding than Claude Security, the next most accurate tool, which ran at 131.87 dollars.
- Parallel agents reason across files to catch business-logic flaws, chained auth bypasses, and cross-service exploits that single-file static analysis cannot see.
- Every finding is reproduced in an isolated sandbox before a fix PR is opened, which sharply cuts the false-positive noise typical of pattern scanners.
- It uniquely caught three critical flaws other tools missed, including a PHP template-injection sandbox bypass and a Spring Kafka deserialization surface.
What to Watch
The strategic question is whether validation-first scanning becomes the category standard. If confirming exploitability in a sandbox is what separates a useful finding from noise, every serious scanner will need a runtime validation stage, and static-only tools will look increasingly thin by comparison. Watch how the cost curve moves as scan volume grows, since agentic runs are more expensive than static passes and the incremental-scan model only partly offsets that. The broader thread worth tracking is the arms race this implies: as AI writes more of the code, the same agentic techniques get pointed at securing it, a dynamic that also runs through our coverage of AI agent security failures. The tools that both write code and prove it safe are converging on the same engineering-agent shape.
Frequently Asked Questions
What is Devin Security Swarm?
It is an agentic security system from Cognition that scans codebases with parallel AI agents, reproduces each vulnerability in a sandbox to confirm it is exploitable, and opens a pull request with a fix. It is built on the same Devin engineering-agent platform.
How accurate is it compared to other scanners?
On a 50-vulnerability benchmark tied to real GitHub Security Advisories, Devin Security caught 72 percent at 90.23 dollars per run, ahead of Claude Security at 68 percent, Codex Security at 48 percent, and Cursor Security at 26 percent.
What makes it different from static analysis?
Static analysis reads one file at a time and flags patterns, producing many false positives. Security Swarm reasons across files and validates each finding by reproducing the exploit in a sandbox, so it catches cross-file logic flaws and files far less noise.
Does it fix the vulnerabilities it finds?
Yes. After confirming an exploit in a sandbox, it writes a patch and opens a pull request for human review, rather than only listing the issue for a developer to fix manually.
How does it handle repeated scans?
The first full scan establishes a baseline, and subsequent scans process only the code that changed. That keeps recurring cost tied to how much the codebase churns rather than its total size.
