AI Powered Coding Assistants: What It Does, What It Costs, and When It's Worth It

Everyone's Using AI to Code — So Why Isn't Shipping Faster?

AI coding assistants have reached near-universal adoption — yet the promised speed hasn't shown up where it counts. According to the Stack Overflow 2025 survey, 84% of developers now use or plan to use AI tools in development, and a separate Black Duck study found 97% of enterprise software teams have adopted assistants. So why isn't anything shipping faster?

The uncomfortable answer comes from telemetry. Faros AI analyzed data from over 10,000 developers and 1,255 teams and found that while more than 75% of developers use AI assistants and report working faster, there's no measurable company-level improvement in delivery velocity or business outcomes. The gains are real — but they're getting stuck somewhere between the developer's keyboard and the production release.

Individual speed is not the problem. Developers reclaim an average of 8 hours per week using AI tools, and 92% of teams report improved productivity or release velocity. The problem is that organizations aren't built to absorb that speed. Here's what the data shows happens when AI-generated code hits the rest of the pipeline:

• PR review time jumped 91% on high-AI-adoption teams, as reviewers wade through 154% larger pull requests with 9% more bugs per developer.
• Roughly 90% of teams encounter issues with AI-generated code, with bottlenecks concentrated in manual review (52%), security testing (51%), and code rework (48%), per the Black Duck research.
• MIT Sloan researchers, studying 100,000+ GitHub developers, found coding activity rose up to +180% with autonomous agents — but those gains attenuated to just +50% more projects and +30% more actual releases.

That last finding is the clearest illustration of the paradox. Writing code is only one stage of software delivery, and it's no longer the slowest one. As the MIT Sloan researchers put it, developers must still test, polish, and iterate toward market fit — and these higher-level tasks remain constrained even when AI slashes the cost of writing code. Faros frames it through Amdahl's Law: a system moves only as fast as its slowest link, and for most teams that link is now review, testing, and release.

There's also a reconciliation worth noting. Black Duck reports 92% of teams see improved productivity, while Faros finds no company-level correlation with delivery outcomes. Both can be true: gains are real at the individual and team level, but they only compound into business results when paired with governance, workflow redesign, and downstream capacity. Teams with full governance are 55% more likely to report major efficiency improvements — yet only 30% actually have it.

The takeaway for anyone evaluating whether AI coding assistants are worth it: individual speed gains evaporate at the organizational level unless you fix what comes after the code. And the same verification mindset applies to the broader agent ecosystem — published research found roughly 26% of agent skills carry at least one vulnerability, which is why SkillzBench is building a free directory where every skill is security-scanned, license-checked, and benchmarked before you connect it to your coding agent. If you want to go further with agent skills, the waitlist is open — browse and test free ones on SkillzBench as the platform launches.

What AI Coding Assistants Actually Do (and What They Cost)

AI coding assistants have quietly become the default way software gets written: Stack Overflow's 2025 survey found 84% of developers now use or plan to use AI tools, up from 76% the year before. But "AI coding assistant" covers three very different generations of tool, and understanding which one you're buying matters more than the brand name on the box.

The first generation is autocomplete — inline suggestions as you type, integrated into editors like VS Code and JetBrains. The second is chat-based help: asking questions, explaining unfamiliar code, and generating snippets in a conversation window. The third and newest is agentic execution, where the assistant plans a task, edits multiple files, runs tests, and iterates until the job is done. MIT Sloan researchers studying 100,000+ GitHub developers found coding activity rose +40% with autocomplete, +140% with sync agents, and +180% with async agents — though, as we'll see later, those gains shrink at the organizational level.

Beyond writing new code, these tools handle the unglamorous work that eats engineering time:

• Upgrading legacy code and translating between programming languages
• Generating documentation and explaining unfamiliar codebases
• Creating test cases and boilerplate that developers dread writing

Because the underlying models are largely shared across tools, hands-on evaluations find the real differentiator is context quality — how well a tool retrieves relevant code from your actual repository, especially at enterprise scale.

Pricing spans a wide range. GitHub Copilot's free tier includes 2,000 completions per month; paid entry points cluster around $10–$39 per user (Copilot Pro, Cursor Pro at $20, Tabnine at $39). At the top end, Claude Code Max and ChatGPT Pro with Codex run $200/month. Notably, the market is shifting from seat-based to usage-based pricing — AWS's Kiro, for instance, charges credits ($20 for 1,000 up to $200 for 10,000), and Replit uses effort-based pricing, a trend market analysts attribute to long-context models making flat per-seat models uneconomical.

For teams extending these assistants with agent skills, verification is worth budgeting for too: published research found roughly 26% of skills carry at least one vulnerability. SkillzBench is building a free answer to that — a directory where every skill is security-scanned, license-checked, and benchmarked so you know what actually helps before you install it.

The Real ROI Levers: Governance, Grounding, and Bottlenecks

Buying an AI coding assistant is easy; turning faster coding into shipped software is where most organizations quietly fail. The research points to three levers that separate teams that see real returns from teams that just see bigger diffs.

The first lever is governance. According to Black Duck's enterprise study, teams with full governance are 55% more likely to report major efficiency gains — yet only 30% actually have it. As Black Duck CEO Jason Schmitt puts it, "speed without governance is a liability, not an advantage."

The second lever is fixing bottlenecks downstream of the editor. MIT Sloan research tracking 100,000+ GitHub developers found coding activity gains of +40% with autocomplete and up to +180% with autonomous agents — but those gains attenuated to just +30% more actual releases. Faros telemetry echoes this: high-adoption teams saw PR review time jump 91% and 9% more bugs per developer, because a system moves only as fast as its slowest link.

The third lever is grounding, not model choice. Since most tools share underlying models, differentiation comes from how well they retrieve context from your actual repository. Agents relying only on local tools like grep and file reads struggle systematically above ~400,000 lines of code, while indexed retrieval made agents 30% cheaper and 38% faster in one benchmark — a Kubernetes task that took ~2 hours with a local-only agent took 89 seconds with indexed search.

And the risks are real enough to demand all three levers:

• ~90% of teams encounter issues with AI-generated code, with bottlenecks concentrated in manual review (52%), security testing (51%), and rework (48%).
• 64% of teams worry about AI assistants introducing security defects — and the most concerned teams are also the heaviest users.
• 84% of developers still want a human in the loop, and 86% believe an AI agent should evaluate AI-generated code.

The same verification gap applies to the agent skills and instruction packages that extend these coding tools — published research found roughly 26% carry at least one vulnerability. SkillzBench is being built to close exactly this gap: a free directory where every skill gets security-scanned, license-checked, and benchmarked so teams can adopt AI with evidence instead of optimism. The lesson from the data is consistent: the ROI shows up where governance, retrieval quality, and review capacity are treated as part of the purchase, not an afterthought.

When It's Worth It — and How to Decide

So you've seen the demos, the pricing tiers, and the productivity claims. The real question isn't whether AI coding assistants work — it's whether they'll work for your team, and whether you can capture the gains before they leak away downstream.

Start with skill level, not vendor hype. Every major tool offers a free tier or low-cost entry — GitHub Copilot's 2,000 free completions per month, Cursor's Hobby plan, Replit's Starter tier — so there's no reason to commit to $10–$39/user/month plans before testing fit. According to hands-on reviews, a sensible pattern is matching the tool to the user: beginners prototype in Replit, intermediate developers graduate to Cursor or v0, and senior engineers get the most from agentic tools like Claude Code.

Then fix your bottlenecks before buying more speed. This is where most organizations quietly lose their ROI. MIT Sloan research tracking 100,000+ developers found coding activity rose dramatically with AI — but those gains attenuated to just +30% more actual releases, because developers still have to test, polish, and iterate toward market fit. Faros telemetry confirms the pattern: on high-adoption teams, PR review time rose 91% and bugs per developer increased 9%.

That means the highest-leverage investments aren't more coding horsepower — they're review capacity, testing automation, and release pipelines. As Faros frames it via Amdahl's Law, a system moves only as fast as its slowest link.

A practical decision framework looks like this:

• Start free, evaluate on your actual codebase, then scale to paid seats only after measurable gains.
• Stand up governance before scaling — teams with full governance are 55% more likely to report major efficiency improvements, yet only 30% have it.
• Reallocate budget toward review, security testing, and merging — the stages where ~90% of teams hit issues with AI-generated code.
• For regulated environments, choose self-hosted, license-aware tools like Tabnine, which offers air-gapped deployment and zero code retention.

Finally, plan for the role shift. 84% of developers still want a human in the loop, and developers expect to spend more time on review and validation (29%), architecture (29%), and security verification (23%), per the same enterprise research. The winners won't be the teams that generate the most code — they'll be the ones that verify it best.

If you're extending coding agents with third-party skills, the same verification logic applies: SkillzBench is building a free directory where every skill gets security-scanned, license-checked, and benchmarked before you deploy it — closing the exact trust gap this research surfaces.

Extending Your Assistant Safely: The Skills Layer

Once you've picked your assistant and seen real gains, the next temptation is extending it. Agent skills — instruction packages that teach Claude Code, Codex, Cursor, or Gemini CLI how to handle specialized workflows — are multiplying fast across GitHub and community repositories. But extending your assistant means trusting third-party code and instructions, and that trust is exactly where the research says teams get burned.

The numbers are sobering. Published research found that roughly 26% of agent skills carry at least one vulnerability, and the broader governance picture is worse: a Black Duck study found ~90% of teams encounter issues with AI-generated code, with bottlenecks concentrated in manual review (52%) and security testing (51%). Meanwhile, 64% of teams worry about AI assistants introducing security defects — and the heaviest users worry the most.

This is the gap between "skills exist" and "skills are safe to use." Right now, nobody can tell you which skills actually help your agent and which quietly introduce risk. Verification is not optional; it's the difference between compounding your productivity gains and importing someone else's vulnerabilities into your codebase. As Black Duck's CEO puts it, "Speed without governance is a liability, not an advantage."

That's the problem SkillzBench (skillzbench.com) is being built to solve. It's a free, independent directory of agent skills designed around a simple principle: verify before you equip. The platform is being built to:

• Verify every skill — pinning each version to a commit SHA, then security-scanning, license-checking, and quality-judging it
• Benchmark each skill in a sandbox with the same model, with and without the skill, publishing the Lift in percentage points so you see what actually helps
• Equip any agent through a free MCP connector, with per-agent loadouts so each agent only sees the skills and tools you chose for it
• Stay current by tracking upstream changes, re-scanning, and re-benchmarking when skills update

A 0–100 trust score drawn from security, quality, license, maintenance, and community signals will sit alongside the Lift score — always kept separate, because money never touches scores. Everything is free: the directory, the benchmarks, the alerts, and the MCP connector.

SkillzBench is pre-launch and actively being built. Join the waitlist to get early access and be first to browse security-scanned, benchmarked skills for your assistant.

ctaText: Join the SkillzBench waitlist — get early access to verified, benchmarked agent skills. socialProofText: Skills, proven. Security-scanned, license-checked, and benchmarked before your agent ever loads them.