A new study published August 5, 2026 measured how well people actually catch dangerous commands when an AI coding agent asks for approval, and the answer is sobering: across more than 40,000 sessions and roughly 409,000 individual approve-or-deny decisions, humans missed about one in three genuine threats. The mean accuracy was 66.3%. If you approve your coding agent's actions under time pressure, this is a direct look at how often that safety net fails.

What Happened

The ScaleX study ran as a browser-based game: players reviewed a stream of agent commands, a mix of routine and malicious, and had to approve or block each one under a clock. The aggregate data quantifies "human-in-the-loop" oversight, the exact model most AI coding tools use when they pause and ask "allow this command?" The finding is that the loop leaks badly, and it leaks worst on the threats that are not obviously destructive.

Why It Matters

Most agentic coding setups treat human approval as the last line of defense. This data suggests that line is thinner than teams assume. The failure maps closely to a known category in the OWASP Top 10 for LLM applications: excessive agency, where an agent is granted more capability than a human can realistically supervise. It also intersects with prompt-injection risk, the subject of years of documented research, because a malicious instruction hidden in a file or web page can produce a command that looks routine at a glance.

Key Details

  • 66.3% mean accuracy across 40,000+ sessions and about 409,000 decisions.
  • 32.9% of sessions ended with a negative score (net harm approved).
  • 7% of players approved every single command, blocking nothing.
  • Miss rates climbed by subtlety: obvious destructive commands 11.7%, persistent mutation 23.8%, exfiltration or code execution 33.4%, scope violations 35.0%.
  • The single most-missed command was npm run analyze at 64.7%, a benign-looking script name that can hide arbitrary execution.

What to Do Next

Do not rely on click-through approval as your only guardrail. A stronger setup for anyone running a coding agent:

  1. Deny by default for whole categories rather than judging each command live. Block network calls, package scripts, and file exfiltration paths unless explicitly needed.
  2. Run agents in a sandbox or disposable container so an approved-by-mistake command cannot touch real credentials or production data.
  3. Treat opaque script names as high-risk. A command like npm run analyze should trigger a look at what the script actually does before approval.
  4. Read your tool's own guidance. The Claude Code security docs and similar pages spell out permission models worth configuring before you start approving on autopilot.

The takeaway for builders: human review is a supplement, not a substitute, for locking down what an agent is allowed to do in the first place.