Can AI Agents Actually Handle Python? What Skills Can and Cannot Do in 2026

The Adoption-Trust Gap: Everyone Uses AI Agents, Few Trust Them

Here's the strange truth of AI coding in 2026: the tools are everywhere, and almost nobody fully believes what they produce. Roughly 90% of professional developers now use AI coding agents at work at least weekly, with 68% using them daily, according to 2026 developer usage data. Yet in the same period, 46% of developers reported distrusting AI accuracy.

That gap between usage and trust isn't irrational. It's grounded in how the tools are measured.

Why the benchmarks flatter the agents

Most headline benchmark scores you see quoted in vendor announcements come from public datasets, and independent research shows these scores overstate real-world capability by 20–50%. When researchers tested agents against internal, private benchmarks instead, the gap shrank to a much more modest 10–16%. The reason is simple: benchmark problem statements typically run over 100 words of carefully specified context, while real developer queries are 10–30 words, vague, and full of unstated assumptions. Agents trained and evaluated on polished problems stumble on the messy ones.

This mismatch shows up in your daily workflow as the "almost right" problem. The generated Python function runs, mostly. The refactor looks clean until an edge case breaks it. Bug fixing alone accounts for roughly 14% of all user queries to chat-based coding agents — much of it time spent correcting output the agent confidently presented as finished.

What this means for how you work

• Treat published benchmark scores as upper bounds, not expectations for your codebase.
• Pilot agents on your real queries and real repositories before standardizing on any tool.
• Budget for review time — the trust gap exists because verification still falls on humans.
• Be equally skeptical of the skills and extensions you bolt onto agents; a large share carry security flaws that make reliability worse, not better.

The same skepticism applies to the broader agent ecosystem. Published audits found that roughly 26% of agent skills carry at least one vulnerability, which is why SkillzBench is building a free directory that security-scans, version-pins, and benchmarks every skill it lists — so you can see measured Lift in percentage points rather than trusting marketing claims. It's one way to close the gap between what agents promise and what they actually deliver, one verified component at a time.

What Agents Genuinely Can Do in Python (and the Harness That Makes the Difference)

Strip away the hype, and the evidence shows AI agents genuinely can handle well-scoped Python work — but only when the surrounding system is built right. The catch is that "built right" has less to do with the model than most people assume.

On clearly defined tasks, today's agents perform strongly. Controlled testing shows top models hitting 66.4% on Terminal-Bench 4.0 and 54.4% on FrontierCode v1.1, with Opus 5.5 completing tasks at $3.97 each in high-effort mode, per published benchmark results. The key word is "well-scoped": these are tasks with clear inputs, clear success criteria, and a single agent working alone.

The more surprising finding is what actually differentiates performance. According to a controlled 14-day comparison, the harness — the system prompt, tools, agent loop, and context management wrapped around the model — matters more than the model itself. The same model (Opus 4.8) achieved the same success rate across three harnesses while burning wildly different amounts of tokens:

• Claude Code: ~742,000 tokens per task
• Codex: ~1,235,000 tokens per task
• Pi: ~237,000 tokens per task — roughly 5x cheaper than Codex for identical results

Initial prompt overhead ranged from ~33,000 tokens down to under 1,000. As the tester put it: "Same model, half the cost in one harness. Yeah, that really happens." The practical advice that follows is blunt: pick the harness first, pick the model second.

This is also where skills enter the picture. The SKILL.md format has emerged as a cross-platform standard, letting instruction packages move between Claude Code, Codex, Cursor, and other harnesses. Skills are how you extend an agent's Python capabilities — testing conventions, packaging rules, deployment recipes — without retraining anything.

But portability cuts both ways. Snyk's audit of 3,984 skills found 36.82% carry at least one security flaw, with 13.4% containing a critical-level issue, and skill submissions surged 10x in just weeks, per the ToxicSkills audit. A skill that helps one harness may be wasteful, risky, or actively malicious in another.

That verification gap is exactly what SkillzBench is being built to close: a free directory that pins every skill to a commit SHA, security-scans it, and benchmarks it in a sandbox with and without the skill to measure real Lift. The model does the coding; the harness and the skills decide whether that coding is reliable, affordable, and safe.

The Documented Limits: Teamwork Failures and Benchmark Inflation

The most impressive AI demos you'll see in 2026 share one trait: they're performed alone, with a detailed prompt, by a single agent. The moment reality deviates from those conditions, performance drops — and the research quantifies exactly how much.

Stanford HAI's CooperBench study, spanning more than 650 real-world software engineering tasks across Python, TypeScript, Go, and Rust, found that two top agents working together lose nearly half their individual capability. The shortfall hits hardest on midrange tasks — precisely where collaboration was supposed to help. Letting agents message each other in real time had "almost no impact."

The failure modes are strikingly human: ignoring direct warnings, overwriting teammates' code, spamming low-value status updates, and failing to follow through on commitments. As Stanford's Hao Zhu puts it, models "do not use language for social action" — they're trained not to, and it shows.

Benchmarks have a second problem: they don't test what users actually write. A Microsoft research paper analyzing roughly 10,000 real queries found that developers typically type 10–30 words, while benchmark problem statements exceed 100 words. That mismatch means public benchmarks overstate real-world capability by 20–50% — narrowing to 10–16% only on private, internal datasets.

The gap concentrates in one area: debugging. Bug fixing accounts for about 14% of real user queries, and the researchers conclude that reported performance "may not translate to real-world settings where users communicate very differently." For deployment planning, that means treating headline scores as upper bounds, not expectations.

Realistic expectations for 2026 look like this:

• Single-agent, well-specified Python tasks: strong and reliable
• Multi-agent teamwork: expect roughly a 50% capability penalty until verification mechanisms mature
• Bug fixing from short, vague prompts: the documented weak spot
• Public benchmark scores: mentally discount by 20–50%

This is also why private, repeatable evaluation matters more than leaderboards. SkillzBench is being built around exactly that principle — testing every agent skill in a sandbox with the same model, with and without the skill, so the measured Lift reflects real conditions rather than inflated scores. If you're equipping agents with skills, measured performance beats advertised performance every time.

The Skills Supply-Chain Problem: One in Four Skills Is a Liability

One in four agent skills is a liability. That is not a rounding error — it is the headline finding from Snyk's audit of 3,984 skills on ClawHub, where 13.4% contain at least one CRITICAL-level security flaw and 36.82% carry at least one flaw at any severity.

The volume problem is accelerating faster than the defenses. Daily skill submissions jumped from under 50 in mid-January 2026 to over 500 by early February — a 10x increase in weeks. Publishing on ClawHub requires only a SKILL.md Markdown file and a GitHub account one week old. There is no code signing, no security review, and no sandbox by default.

The attacks are not theoretical. Mitiga Labs demonstrated a proof-of-concept in which a seemingly legitimate skill silently exfiltrated an entire local codebase to an attacker-controlled public repository, signed under the author of Cursor Agent, with an empty audit log. The researchers noted the chilling detail: "we just told Cursor to help us make the instructions more silent and require fewer interactions – and so, Cursor happily delivered."

Stars and upvotes are not trust signals in this ecosystem. Snyk identified 76 confirmed malicious payloads, with 8 still live on ClawHub at publication. Every confirmed malicious skill contained malicious code patterns, and 91% simultaneously employed prompt injection. Hardcoded secrets appeared in 10.9% of all skills — and in 32% of confirmed malicious samples.

Red flags to watch for before installing any skill:

• Instructions that explicitly silence or minimize user-facing prompts and confirmations
• Allowlists that exclude specific files, directories, or network destinations
• Hardcoded credentials, tokens, or API keys embedded in SKILL.md or supporting files
• Network calls to unfamiliar domains, especially during routine operations
• A recent publish date combined with no version pinning to a commit SHA

Snyk's CRITICAL-level detectors achieve 90–100% recall on confirmed malicious skills with 0% false positives on the skills.sh top-100, which suggests automated scanning is a viable baseline — but only if it runs before the skill touches a real machine. That is the gap SkillzBench is built to close: pinning every version to a commit SHA, security-scanning every submission, and letting users toggle skills on or off per agent so nothing loads unless it has been verified first.

A Practical Playbook: Verify Skills, Match Tools to Roles, Benchmark on Your Own Code

The gap between what AI agents score on public benchmarks and what they actually deliver on your codebase is the single most important number to understand before you commit to a workflow. Microsoft's research shows that traditional benchmarks overestimate agent capabilities by 20–50% on public datasets — but that gap narrows to just 10–16% when agents are tested on internal, private code. That's why the most reliable evaluation is the one you run yourself.

Step 1: Audit every skill before it touches your environment. Snyk's audit of nearly 4,000 agent skills found that 36.82% carry at least one security flaw at any severity, and 13.4% contain a critical-level issue. Never install a skill based on stars or upvotes alone — read the SKILL.md instructions line by line and flag any language that silences the agent or allowlists commands.

Step 2: Choose the harness by workflow, not hype. The same model can burn three times as many tokens depending on which harness wraps it — one controlled test found Pi used roughly 237,000 tokens per task while Codex used over 1.2 million on identical tasks with the same success rate. Match tools to roles: IDE-first tools for daily editing, terminal-first agents for long delegated tasks.

Step 3: Avoid unverified multi-agent setups. Stanford's CooperBench study found that two top agents working together lose nearly half their individual capability, and real-time messaging between them had almost no impact on results. If you do run multi-agent workflows, add verification mechanisms and periodic integration checks.

Step 4: Benchmark on your own code — that's where the truth lives.

• Run agents on real internal tasks, not synthetic benchmark problems — real developer queries are typically 10–30 words, while benchmark prompts exceed 100.
• Measure Lift (percentage-point improvement) with and without each skill on the same model and the same task set.
• Pin every skill to a commit SHA so a silent upstream update can't change behavior mid-evaluation.
• Re-benchmark periodically; skills change, and a tool that helped last month may add noise this month.

SkillzBench is building exactly this verification and benchmarking layer — a free directory that pins every skill to a commit, security-scans and license-checks each one, and publishes measured Lift scores alongside trust ratings. Its free MCP server lets you toggle only the skills you've verified for each agent, so nothing unvetted reaches your codebase.