Is AI failing as a software engineer?

The Trust Paradox: Everyone Uses AI, Almost Nobody Believes It

Every morning, millions of developers open an AI assistant before they've opened their coffee. Yet when Stack Overflow asked whether they trust what that assistant produces, only 29% said yes — down from 40% the year before, according to industry survey data.

The adoption numbers tell one story. Per compiled statistics, 84% of developers use or plan to use AI tools, up from 76% in 2024, and 51% of professionals use them daily. DORA's 2025 findings show 90% of software teams use AI at work daily, with respondents spending a median two hours per day on AI-assisted work — structural integration, not an experiment.

The trust numbers tell the opposite story. Only 3% of developers report high trust in AI accuracy, while 46% actively distrust it. Favorable sentiment toward AI has fallen to 60%, down from over 70% in 2023–2024.

The evidence behind that distrust is concrete:

• 45% of AI-generated code introduced an OWASP Top 10 vulnerability in Veracode testing across 100+ models
• AI-authored pull requests produced 10.83 issues per PR versus 6.45 for human-only code — roughly 1.7× more, per CodeRabbit's analysis of 470 open-source PRs
• 66% of developers cite "almost right, but not quite" output as their biggest frustration

Here is the paradox in plain terms: developers keep using tools they increasingly don't believe. That's not irrationality — it's a rational response to a tool that genuinely accelerates the first 70% of a task while quietly degrading the last 30%, where edge cases, architecture, and security live.

The real question for teams is therefore not whether to use AI. Banning it doesn't work anyway — up to 57% of developers admit using unsanctioned assistants despite prohibitions, driving unmonitored "Shadow AI" risk, as OX Security notes. The question is why AI keeps letting teams down in predictable, repeatable ways — and what a mitigation strategy that actually works looks like.

Part of the answer involves extending agents with verified skills and tools rather than whatever surfaces first in a search — the discover, verify, and benchmark workflow SkillzBench is building exists precisely because unvetted components inherit the same trust problem this data describes. The rest of the answer lies in the failure modes themselves.

Trust has fallen 11 points in a single year while daily usage climbed. That gap is the whole story — and closing it starts with understanding exactly where AI breaks.

Four Failure Modes That Keep Showing Up in the Research

The evidence against AI as an autonomous engineer doesn't come from anecdotes — it comes from controlled testing, large-scale code analysis, and telemetry across thousands of developers. Four failure patterns appear again and again across independent studies.

Security vulnerabilities lead the list. Veracode testing across more than 100 models and 80 tasks found that 45% of AI-generated code introduced an OWASP Top 10 vulnerability, with Java faring worst at roughly 72% failure — and larger models were not consistently safer (Uvik's compilation). Apiiro's telemetry adds that AI commits merged 4× faster while showing 322% more privilege escalation paths and 2.5× more critical vulnerabilities (Cerbos reports).

Issue density per pull request is measurably worse. CodeRabbit's analysis of 470 open-source PRs found AI-authored changes produced 10.83 issues per PR versus 6.45 for human-only code — roughly 1.7× more problems per change, including 75% more logic errors and up to 2.74× more security issues (CodeRabbit's report). Notably, no issue category was uniquely AI's; AI simply makes the same mistakes more often and at larger scale.

Hallucinated packages create real supply-chain risk. A USENIX Security 2025 study identified 205,474 unique hallucinated package names across 576,000 generated samples, with open models hallucinating dependencies 21.7% of the time (Uvik's statistics review). Attackers exploit this through "slopsquatting" — registering fake packages under names models invent, so the next developer who trusts the suggestion installs malware. GitHub now lists suspicious packages among its official review checkpoints (GitHub's documentation).

Code quality is drifting downward. GitClear found refactoring collapsed from ~24% of code changes to under 10%, while copy/paste rates rose from 8.3% to 12.3% between 2021 and 2024 (DigitalApplied's analysis).

Underlying all four is the "productivity placebo" from METR's 2025 randomized trial: experienced developers were 19% slower with AI while believing they were 20% faster. As one researcher put it, LLMs hijack the brain's reward system — you get the feeling of achievement without the heavy lifting.

The mitigation pattern that emerges is consistent:

• Treat AI output as draft code requiring mandatory review, especially for business logic
• Verify every dependency AI suggests before merging — existence, maintenance, provenance
• Automate guardrails rather than relying on manual review alone
• Govern instead of banning — bans push usage into unmonitored "Shadow AI"

The same verification discipline extends to the skills and tools agents load: research found ~26% of agent skills carry at least one vulnerability, which is why SkillzBench is building security scanning, license checks, and sandboxed benchmarking into its free skills directory before agents ever touch unvetted instructions.

Why AI Fails Autonomously but Wins With a Human in the Loop

AI doesn't fail randomly — it fails predictably, and the pattern matters more than the anecdotes. When researchers systematically examined the evidence, the same root causes kept surfacing, and they're structural, not fixable with a better prompt.

A systematic literature review of 72 studies traces bugs in AI-generated code to two origins: flaws inherited from buggy public training data scraped from GitHub and Stack Overflow, and inherent model limitations — hallucinations, weak semantic reasoning, and an incomplete grasp of what the program is actually supposed to do. The model doesn't understand your repository's intent; it predicts plausible-looking text.

This is the heart of what practitioners call the "70% problem": AI gets you most of the way quickly, then falls apart on the final stretch of edge cases, architecture, and cleanup. As one widely shared analysis puts it, "an LLM has no intent, which is why the final 30% always falls apart" (Cerbos). CodeRabbit's analysis of 470 open-source pull requests quantifies the fallout: AI-authored changes produced 10.83 issues per PR versus 6.45 for humans — roughly 1.7× more — including 75% more logic and correctness errors (CodeRabbit).

Yet the gains are real — they just live in specific places. Per the Uvik synthesis, benefits concentrate in:

• Greenfield projects with no legacy constraints
• Boilerplate, scaffolding, and repetitive structure
• Test generation and documentation
• Junior developers, who gain the most (a Microsoft/Harvard/MIT field study found seniors saw little to no speed-up)

For experts in mature codebases, the effect can invert entirely: METR's randomized trial found experienced developers were 19% slower with AI while believing they were 20% faster — a productivity placebo driven by how satisfying generated code feels.

The takeaway isn't to abandon AI; it's to scope it. Treat AI output as draft code requiring mandatory review, feed it repo-specific context deliberately, and verify everything it suggests. The same discipline applies to the skills and tools you attach to coding agents — validation before deployment is the whole game, and it's the discover, verify, and benchmark workflow SkillzBench is building for agent skills. Human-in-the-loop isn't a limitation on AI; it's the condition under which AI actually works.

The Mitigation Playbook: Verify, Guardrail, Govern

Every failure mode in the research has a matching countermeasure — and none of them is "stop using AI." The evidence points to a systems problem: AI output is draft code, and drafts need review pipelines, not blind trust. Here is the playbook that falls directly out of the data.

Start with the last 30%. The well-known "70% problem" holds that AI gets you most of the way quickly, then falls apart on edge cases, architecture, and cleanup — because, as one analysis puts it, "an LLM has no intent." Make human review mandatory for that final stretch, especially business logic, where CodeRabbit's analysis of 470 pull requests found AI-authored changes carried 75% more logic and correctness issues than human code.

Verify every dependency. Researchers identified 205,474 unique hallucinated package names across 576,000 generated samples, and attackers exploit these fabrications through "slopsquatting" — registering the fake names as real, malicious packages. GitHub's own guidance says to check every AI-suggested dependency for existence, maintenance, licensing, and provenance before merging. No exceptions.

Automate the guardrails, because manual review alone cannot keep pace with 3–4× more commits:

• Run SAST on AI-generated code — but choose tools carefully, since legacy scanners generate 76%+ false positives, which trains developers to ignore alerts.
• Enforce policy-as-code so exception handling, credential handling, and dependency rules are checked automatically at PR time.
• Write tests for control flow, not just happy paths — AI produces "surface-level correctness" that skips protective branches.
• Feed the model repo-specific context: your README, docs, and recent PRs, plus explicit instructions on which sources to trust.

That last point matters more than teams realize. CodeRabbit found AI drifts toward generic defaults when it lacks local context, recreating outdated security patterns from training data instead of your repo's idioms. The same verification principle extends to whatever else you bolt onto your agents — skills, MCP tools, third-party instructions all deserve the same scrutiny as the code they produce. SkillzBench is building exactly that kind of discover, verify, and benchmark workflow for agent skills, so teams can test whether an add-on actually helps before an agent ever touches it.

Finally, govern, don't ban. Up to 57% of developers admit using unsanctioned AI assistants despite prohibitions, and bans simply push that usage into unmonitored "Shadow AI." Approved tooling with guardrails beats prohibited tooling with none — provide sanctioned assistants, corporate system prompts, and documented review checklists, and you keep the velocity while containing the risk.

Skills, proven. If you're equipping agents with skills and tools, browse SkillzBench's free directory and join the waitlist to see verified, benchmarked agent skills before your agents do.

Extending the Same Discipline to Agent Skills and Tools

You verify AI-generated code before merging it — but what about the skills and tools your coding agent loads from strangers on the internet? The same blind trust that produces hallucinated packages applies to the agent ecosystem itself, and the stakes are rising fast.

The scale of the trust problem is well documented. A USENIX Security 2025 study identified 205,474 unique hallucinated package names across 576,000 generated samples, and attackers actively register those fake names — a technique called "slopsquatting" — to inject malicious code. GitHub's own guidance now tells developers to scrutinize every AI-suggested dependency for existence, maintenance, licensing, and provenance before merging.

Agent skills and MCP tools deserve identical scrutiny. These instruction packages and connectors — used with Claude Code, Codex, Cursor, and Gemini CLI — can instruct your agent to run commands, call APIs, and touch your repository. Yet most teams install them with less diligence than they'd apply to a single npm package. Security researchers warn that models replicate historical vulnerabilities and hallucinate dependencies, and real incidents like the poisoned Amazon Q extension in 2024 show that compromised tooling can delete files and shut down cloud infrastructure.

A sane verification workflow for agent extensions looks like this:

• Pin every skill or tool to a specific commit SHA, so you know exactly what version you audited
• Run security scans before an agent ever executes the skill's instructions
• Check the license — an instruction package with unclear licensing can create compliance risk
• Benchmark the skill with and without it in a sandbox, so you know it actually helps rather than guessing
• Track upstream changes and re-scan when the source updates

This is precisely the gap SkillzBench is being built to close. The platform is designed as a free, independent directory that continuously discovers skills from GitHub, then verifies each one — commit-pinned, license-checked, security-scanned, and quality-judged — before benchmarking it in a sandbox to publish its measured Lift. A single free MCP connector then lets you equip any agent with only the skills you've chosen and vetted, so your agent never sees unvetted instructions at all.

The evidence says manual review alone won't scale. Apiiro found AI commits merge 4× faster while bypassing review, and up to 57% of developers admit using unsanctioned tools despite prohibitions. Verification has to happen before the agent runs, not after the damage. If that principle works for code — and the research strongly suggests it does — it works for everything your agent touches.

SkillzBench is in active development ahead of launch. Join the waitlist at skillzbench.com to get early access to the directory, the free MCP connector, and benchmark results as they publish.