AI Coding Agents: What It Does, What It Costs, and When It's Worth It

The Adoption Rush Meets the Productivity Paradox

Enterprise software teams have never adopted a technology faster — or with less proof it's working. AI coding agents have become the default way professional developers write software, and the money confirms it: coding pulled in $4 billion in enterprise AI spend in 2025, the single largest category in the entire AI application layer, with code completion alone accounting for $2.3 billion, according to Menlo Ventures' enterprise analysis.

The adoption numbers are staggering. JetBrains' 2026 developer survey of 15,000+ developers found that 90% of professional developers now use AI coding agents at work at least weekly, with 68% using them daily. A Harvard study of 700+ firms found 95% had already implemented them. This is not an experiment anymore; it is infrastructure.

Here is the problem: two of the largest empirical studies ever run on this question found no measurable improvement in the software companies actually delivered.

The Harvard/Jellyfish study — built on 300 million work events across 700,000+ employees — found a 30% increase in lines of code and 23% more pull requests, but no statistically significant change in completed software features. Resolution rates for Issues and Epics simply didn't move. Meanwhile, Faros AI's analysis of 10,000+ developers and 1,255 teams found no significant correlation between AI adoption and company-level improvements in throughput, DORA metrics, or quality KPIs. As Faros put it: developers say they're working faster, but companies are not seeing measurable improvement in delivery velocity or business outcomes.

So where do the gains go? Both studies converge on the same answer — they get absorbed downstream:

• High-AI-adoption teams merge 98% more PRs, but PR review time balloons by 91%
• Review process time rose 49% firm-wide, and PRs with changes requested nearly doubled
• Average PR size grew 154%, with 9% more bugs per developer

In other words, agents are generating more code faster, and humans are spending the savings reviewing it. The bottleneck didn't disappear — it moved from writing to checking. That's a genuinely uncomfortable finding for anyone who just signed an enterprise contract on the promise of doubled engineering output.

This is also why the quality of what you bolt onto an agent matters as much as the agent itself. Published research has found roughly 26% of agent skills carry at least one vulnerability, which is why SkillzBench is building a free, independent directory that security-scans, license-checks, and benchmarks every skill before it earns a listing. If agents are writing more code than ever, the tooling around them has to raise its game.

Which raises the question every buyer should be asking: if nearly everyone is purchasing, why aren't outcomes moving? The rest of this article answers that — what agents actually do, what they really cost, and the conditions under which the spend pays off.

What AI Coding Agents Actually Do (and What They Don't)

Most teams now use AI coding tools daily, yet confusion about what "agentic" actually means keeps slowing adoption decisions. The label gets slapped on everything from autocomplete to fully autonomous code-writing systems, and the distinction matters for cost, risk, and where these tools genuinely pay off.

Anthropic's engineering guidance draws a sharp line: workflows follow predefined code paths, while agents let LLMs dynamically direct their own processes and tool usage. In practice, that means assistants auto-complete code primarily authored by humans, whereas agents primarily write and submit code autonomously based on prompts.

That autonomy comes with a cost. Anthropic explicitly warns that "agentic systems often trade latency and cost for better task performance." The recommendation: reach for the simplest solution that works, often a single LLM call with retrieval, before paying the agentic premium.

The agent label now stretches across the entire software development lifecycle, not just code generation. According to Menlo Ventures' 2025 enterprise AI analysis, specialized tools have emerged for nearly every stage:

• Prototyping (Lovable)
• Refactoring (OpenHands)
• Design-to-code conversion (Weaver)
• QA and testing (Meticulous)
• Pull request automation (Graphite)
• Deployment and release (Harness)

This breadth explains why coding captured $4B in enterprise spend in 2025, the largest single category across the entire AI application layer.

Anthropic's framework for evaluating agent fit is worth memorizing: agents suit open-ended problems where it's difficult or impossible to predict the required number of steps. Coding qualifies because output is verifiable through automated tests, agents can iterate on test feedback, and quality is objectively measurable.

Predictable, repeatable workflows don't need that autonomy. A deployment pipeline with fixed steps, a linting check, or a standard code review template all run better as deterministic scripts. Paying agentic costs (higher token spend, longer latency, compounding error risk) for tasks a simple workflow handles is the most common overshoot Anthropic sees in customer implementations.

Before deploying an agent, ask two questions: Can success be verified automatically? Is the task genuinely open-ended, or am I just adding autonomy for its own sake? If both answers are yes, an agent likely fits. If either is no, a workflow or a well-prompted assistant will do the job at a fraction of the cost and risk.

For teams extending agents with third-party skills, vetting matters more than the underlying model. Published research found roughly 26% of agent skills carry at least one vulnerability, which is why SkillzBench is building a free skills directory and MCP connector that pins versions, security-scans, and benchmarks every skill in a sandbox before you load it.

The Real Cost Picture: Subscriptions Are Only the Beginning

Here's an uncomfortable truth about AI coding agent budgets: the sticker price is the smallest line item. Despite coding agents capturing $4 billion in enterprise spend in 2025 — the largest category in the entire AI application layer, according to Menlo Ventures' enterprise AI analysis — no vendor publishes pricing that reflects what adoption actually costs. That's not an accident; it's a structural feature of the market.

The research on this point is refreshingly honest about what it doesn't know. None of the major studies provide specific subscription tiers, token costs, or per-seat pricing. What they do provide is a clear picture of the qualitative cost structure — and it has three layers that most buyers underestimate.

Layer one: integration. Market analysts at MarketsandMarkets flag "high implementation and integration costs" as an explicit market restraint, and nearly 60% of enterprises cite data governance and compliance concerns as adoption barriers. Connecting an agent to your codebase, CI pipeline, and security posture takes real engineering time before anyone writes a line of AI-assisted code.

Layer two: agentic compute overhead. Anthropic's own engineering guidance warns that "the autonomous nature of agents means higher costs, and the potential for compounding errors." Agents trade latency and cost for task performance, iterating across many steps where a single LLM call would do. Anthropic recommends starting with "the simplest solution possible" — paying agentic prices for tasks a workflow can handle is pure waste.

Layer three — the big one: the review bottleneck. This is where coding speed gains go to die. A Faros AI study of 10,000+ developers found high-AI-adoption teams merge 98% more PRs but face 91% longer PR review times, plus 9% more bugs per developer and 154% larger PRs. The Harvard/Jellyfish study of 700+ firms reached the same conclusion: a 30% increase in lines of code with no statistically significant change in completed software features. Efficiency gains are "absorbed by downstream constraints in the production process."

• Integration and governance setup: engineering time plus compliance review before value appears
• Autonomous compute costs: agents iterate repeatedly, multiplying token spend per task
• Review capacity: 91% longer PR review times mean hiring or reallocating senior reviewers
• Quality overhead: larger PRs and more bugs shift cost from writing code to verifying it

The practical takeaway: budget for the whole pipeline, not the subscription. If your reviewers drown, your agents are just expensive text generators. And since agent output quality depends heavily on what skills and tools you attach to them, vetting matters — published research found roughly 26% of agent skills carry at least one vulnerability, which is why SkillzBench is building a free, benchmarked skills directory and MCP connector that scan and verify every skill before it reaches your agents.

When Agents Are Worth It: A Value Test for Your Team

The uncomfortable truth about AI coding agents is that 90% of professional developers now use them at work at least weekly, yet two of the largest empirical studies on the technology found no statistically significant improvement in completed software features at the company level.

That gap between individual enthusiasm and organizational ROI is exactly the question a value test needs to answer.

Anthropic's engineering team identifies coding as a prime agent domain for one specific reason: code is verifiable through automated tests, and agents can iterate against that feedback. The same logic disqualifies most other tasks.

If your use case cannot be tested automatically, you are paying agentic costs for outputs that require human review to validate. The research is blunt about what happens when verification is missing: high-AI-adoption teams merge 98% more pull requests but face 91% longer PR review times, and firms see a 30% jump in lines of code with no measurable change in shipped features.

Before deploying an agent, ask whether a workflow or a single LLM call would suffice. Anthropic's guidance is explicit: "the most successful implementations weren't using complex frameworks" but simple, composable patterns. Agents trade latency and cost for autonomy, and that tradeoff only makes sense on open-ended problems where the number of steps cannot be predicted.

Faros found that companies actually seeing organizational gains share a common foundation. Without these, individual speed gains simply pile up in review queues:

• Workflow design — restructure processes so agents slot into defined handoffs rather than generating unbounded output
• Governance — set policies for what agents may commit, merge, or deploy without human approval
• Infrastructure — provision sandboxes, test runners, and observability so agent work is reproducible
• Training — teach teams prompt craft, review heuristics, and failure-mode recognition
• Cross-functional alignment — ensure product, security, and ops all understand what agents are touching

Most organizations, Faros notes, rely on unstructured bottom-up experimentation instead — which explains why so many see local speedups without firm-level payoff.

Once the foundation is in place, teams often layer in external agent skills to automate adjacent workflow steps. Published research found roughly 26% of agent skills carry at least one vulnerability, which makes vetting non-optional. A free skills directory that pins every version to a commit SHA, scans for security issues, and benchmarks each skill in a sandbox (such as the one SkillzBench is building, with its free MCP connector for toggling skills per agent) lets teams extend capability without inheriting hidden risk.

The value test is straightforward: verifiable task, simplest viable architecture, and the five enablers in place. Miss any one and you are buying individual velocity that the organization cannot absorb.

Getting Started: Pilot, Govern, and Extend Safely

The tools you pick today may not be the tools you keep tomorrow — and that's exactly why your first move should be a pilot, not a purchase order. JetBrains' 2026 survey of 15,000+ developers shows how volatile this market is: Claude Code now leads at 39% work adoption, GitHub Copilot has fallen from 29% to 21%, and Codex grew roughly fivefold in six months, from 3% to 16% source. Committing to a vendor on brand legacy alone is a losing bet.

Pilot on current fit, not incumbency. Run two or three agents against the same real tasks for 30 days, measure what matters, and re-evaluate quarterly. Anthropic's own engineering guidance recommends finding "the simplest solution possible" — often a single LLM call with retrieval beats a full agent, and agents only earn their higher cost on open-ended, unpredictable work source.

Before you scale, plan review capacity. Two independent large-scale studies converge on the same bottleneck: high-adoption teams saw PR review time climb 91% while merging 98% more PRs source, and a Harvard study of 700+ firms found review process time up 49% with no significant change in completed features source. Your agents will write more code; someone still has to read it.

A safe rollout covers three fronts:

• Governance first: nearly 60% of enterprises cite compliance and data governance concerns as adoption barriers, so set data-handling and approval rules before broad deployment source.
• Review budget: staff reviewers and adjust PR-size limits in anticipation of larger, faster-moving changesets.
• Extension vetting: screen third-party skills and plugins before agents load them.

That last point deserves real caution. Published research found roughly 26% of agent skills carry at least one vulnerability — and as agents increasingly pull in third-party instruction packages, unvetted extensions become a quiet attack surface. This is why SkillzBench is building a free, independent skills directory plus an MCP connector that security-scans, benchmarks, and pins every skill version to a commit SHA, so your agents only see what you've explicitly vetted and chosen.

Extend gradually from there. Start with verifiable, test-covered tasks where output quality is objectively measurable, then broaden scope as your review process keeps pace. Piloted deliberately, governed properly, and extended with vetted tooling, AI coding agents can deliver real speed — without turning your codebase into an unreviewed flood.