Coding AI: What It Does, What It Costs, and When It's Worth It

The Productivity Paradox: Why Faster Coders Aren't Faster Teams

Coding AI has achieved near-universal adoption: 84% of developers now use or plan to use AI tools, and 51% of professional developers reach for them daily. The assumption that follows is straightforward—faster developers should mean faster teams. The evidence tells a far less comfortable story.

When Faros analyzed telemetry from 22,000 developers across more than 4,000 teams, the results were striking. Epics per developer rose 66% and code tasks climbed 210%, yet production incident probability per merged PR more than tripled. Bugs per developer increased 54%, unreviewed code reaching production jumped 31%, and PR review time stretched 91% longer. The output looked impressive on dashboards; the delivery pipeline quietly buckled underneath it.

The perception gap makes this worse. In a July 2025 RCT, experienced open-source developers using AI tools were 19% slower—while believing they were roughly 20% faster. Developers feel more productive, report higher satisfaction, and yet organizational throughput, DORA metrics, and quality KPIs show no significant correlation with adoption intensity. Faster coders aren't faster teams.

The damage follows a predictable pattern:

• PR review queues swell as AI-generated code arrives faster than reviewers can assess it
• Average PR size grows 51.3%, making each merge harder to scrutinize
• Bugs per developer rise 54% even as task counts double
• Incidents per merged PR more than triple, eroding the reliability gains the speed was supposed to deliver

The bottleneck simply shifts downstream. When review capacity, testing infrastructure, and release pipelines can't absorb the new velocity, AI-driven gains evaporate—and the cost of that evaporation shows up as production incidents, not sprint velocity.

This is the core problem the rest of this article addresses: unmeasured AI adoption creates the illusion of progress while quietly degrading delivery. Developers experience a productivity boost that the organization never sees, while engineering leaders inherit a codebase that is larger, harder to review, and more prone to failure. The tools aren't broken—the measurement gap is. And for teams wiring agent skills into their coding workflows, that gap extends to every unvetted component in the pipeline—roughly 26% of which carry at least one vulnerability, which is exactly why SkillzBench is building a free directory and MCP connector to pin, scan, and benchmark every skill before it touches production.

What Coding AI Actually Does Well (and Where It Breaks Down)

Most coding AI wins happen in the boring middle of software work, not the hard parts. The pattern is consistent across independent telemetry and vendor research: tools deliver reliable value on routine, well-scoped tasks and lose time, or introduce risk, on anything that requires judgment about system design or novel problem-solving.

Where the value is real:

• Boilerplate generation: CRUD endpoints, configuration files, and standard patterns that follow established conventions
• Test scaffolding: writing initial test cases, generating fixtures, and covering obvious edge cases
• Documentation: docstrings, README files, and inline comments from existing code
• Explaining unfamiliar code: summarizing what a function or module does in natural language
• Repetitive refactoring: renaming, formatting, and mechanical code transformations across a codebase

These are tasks where the "correct" answer is well-defined and the model can pattern-match against millions of similar examples in its training data.

Where it breaks down is where the cost of being wrong is highest. Architecture decisions, complex debugging across distributed systems, and novel algorithmic problems require understanding context the model doesn't have, and documented hallucination rates as high as 46.15% mean the tool can confidently produce code that compiles, passes basic checks, and ships a silent defect into production.

That hallucination risk is not abstract. Faros's 2026 telemetry across 22,000 developers found that production incident probability per merged PR more than tripled in high-adoption teams, with bugs per developer up 54% and unreviewed code reaching production up 31%. The code got written faster. It just didn't get checked carefully enough.

There's also a growing non-developer use case worth flagging. MIT Sloan's Rama Ramakrishnan argues that agentic tools like Claude Code are equally valuable for knowledge work beyond programming: they offer file-based "memory," can automate repeatable workflows, and run tasks in parallel. His example: producing ten client briefings simultaneously. For business leaders, this means the same tool a developer uses to refactor code can automate a quarterly report or a competitive analysis, provided the workflow is repeatable enough to specify clearly.

The practical takeaway is to match the tool to the task type. Use it where the pattern is well-established and the output is easy to verify. Keep humans in the loop where the problem is novel, the blast radius is large, or the cost of a hallucinated answer compounds over time.

The Real Cost: Sticker Price vs. Total Ownership

The per-seat price is the least important number on the invoice. List rates of $10–$22.80 per user per month look trivially cheap against a $150,000 engineer's salary, but ROI analysis shows a 100-developer organization spending $400K–$600K per year across multiple tools — before a single API call is billed.

The gap between sticker price and real spend comes from usage-based overages. Heavy Cursor users report surprise bills of $200–$1,400+ per month, and pricing comparisons show Copilot credit overages can push large deployments above $1,900 monthly. As models handle longer contexts, utilization climbs — and enterprises are increasingly shifting from seat-based licensing to usage-based billing, with Copilot moving to $0.01-per-credit charges in June 2026.

Several cost factors deserve a line in your budget model:

• Seat premiums vary widely: Cursor Standard runs roughly 68% higher per seat than Copilot Business at list price, and premium tiers range from $60 to $200 per user monthly.
• Usage overages for power users can dwarf the base subscription by 10x or more.
• Multi-tool sprawl is common — in one survey, 29 of 99 developers ran Claude Code, Cursor, and Copilot simultaneously, tripling license costs.
• Shadow AI spend via personal API keys sits outside procurement entirely, invisible until finance audits card statements.

The honest way to model spend is to work backwards from actual usage, not forwards from list price. Estimate your distribution of light, moderate, and heavy users; apply overage rates to the heavy tail; and add API costs separately, since most organizations' published totals exclude them. A $19/month tool that saves a $150,000 engineer two hours a week recovers roughly $7,500 in productivity per year — but only if that time savings is real and measured, not assumed.

That last point matters more than any pricing table. McKinsey research finds organizations with structured measurement programs capture 3–4x more value from AI tools than those that buy licenses and hope. Cost control and value capture are the same exercise: track per-user spend, adoption cohorts, and delivery outcomes together, and the budget conversation becomes evidence instead of guesswork.

One more hidden cost is risk-based rather than financial. Teams increasingly extend their coding agents with community skills and extensions, and published research found about 26% of agent skills carry at least one vulnerability. A compromised skill can cost far more than any overage bill. SkillzBench is building a free directory that pins, validates, and security-scans agent skills before they reach your agents — worth watching if you're equipping tools like Cursor or Claude Code on a budget.

When Coding AI Is Worth It: The Measurement-Driven Playbook

The difference between teams that extract real value from coding AI and teams that burn budget often comes down to one thing: measurement discipline. McKinsey research finds that organizations with structured measurement programs capture 3–4x more value from AI tools than those that adopt without instrumentation.

Measurement also pays for itself in hard dollars. One software company documented $4.1 million in savings from productivity improvements achieved through measurement-driven course correction—evidence that visibility into what's working (and what isn't) compounds quickly.

If you're setting expectations for a pilot, plan on a 3–6 month window. GitHub's enterprise research indicates measurable returns typically appear within that timeframe when adoption is structured and tracked.

But here's the part most teams get wrong: tool choice matters far less than workflow design. Teams achieving 25–30% gains didn't just add AI to their existing process—they rebuilt workflows around AI capabilities. Bolting a tool onto a legacy pipeline is how you end up with the productivity paradox.

Downstream bottlenecks are the silent killer of AI ROI. When code generation speeds up but review capacity, test stability, and release pipelines don't, the bottleneck simply shifts. Faros telemetry shows high-adoption teams saw PR review time rise 91% and unreviewed code reaching production climb 31%—negating much of the upstream gain.

Security and license risk deserve first-class treatment in your selection criteria, not a footnote. Published research finds that roughly 26% of agent skills carry at least one vulnerability, and hallucination rates in code generation have been recorded as high as 46.15% in controlled studies.

Here's a practical decision framework before scaling any coding AI investment:

• Instrument PR throughput, cycle time, bug rates, and review load before and after adoption
• Budget for total cost of ownership—usage-based overages can run $200–$1,400+/month for heavy users
• Fix review capacity, test brittleness, and release pipelines before scaling generation speed
• Validate every AI-supplied component—agent skills included—for security, license, and quality risk
• Run a 3–6 month pilot with clear success metrics before committing to org-wide rollout

On that last point: agent skills (SKILL.md instruction packages for Claude Code, Cursor, Codex, and similar tools) are a fast-growing way to extend coding AI, but their quality and safety vary wildly. SkillzBench is building a free, independent directory that security-scans, license-checks, and benchmarks each skill's actual Lift in a sandbox—so you can validate before you equip.