Can we trust AI agents?

The Trust Gap: Why Agent Skills Are a First-Class Attack Surface

If you installed an AI agent skill in the past month, the odds are uncomfortably high that you also installed a vulnerability. Snyk's audit of 3,984 agent skills found that 13.4% carry at least one critical security issue, and 36.82% carry some issue at all. This isn't theoretical risk — it's an ecosystem already under attack.

What makes skills so dangerous is what they inherit. A skill runs with the full permissions of the agent hosting it: shell access, file system read/write, credential access, and persistent memory. As Unit 42 put it, the lack of isolation between skill logic and agent authority means installation grants complete control over the agent's identity. Unit 42 also found that roughly 17% of early OpenClaw skills carried malicious payloads, and that five malicious skills passed VirusTotal and ClawScan screening between February and May 2026.

The barrier to entry is almost nonexistent. Publishing a skill requires nothing more than a SKILL.md file and a week-old GitHub account — no code signing, no security review, no sandbox by default. Marketplace trust signals make things worse:

• Install counts can be inflated by a single scripted HTTP request, with no rate limiting or token validation.
• Security scans run only at creation and at a popularity threshold, so "clean" badges go stale while repositories change.
• Silent name-collision overwrites and blind bulk updates let a benign skill turn malicious on any update cycle.
• 10.9% of ClawHub skills contain hardcoded secrets.

The takeaway is blunt: treat every skill as untrusted code, because that is exactly what it is. This is the gap SkillzBench is built to close — pinning every skill version to a commit SHA, continuously re-scanning upstream changes, and letting you equip each agent with only the skills you explicitly choose.

Why the Signals You Trust Are Spoofable: Install Counts, Badges and Scanners That Fail

That "10,000 installs" badge on a skill? It may have cost the uploader a single line of code. When security researchers started pulling apart how agent skill marketplaces actually work, they found that nearly every trust signal users rely on can be faked, gamed, or simply go stale.

Orca Security demonstrated that install counters are driven by unauthenticated GET requests with no rate limiting or token validation — meaning a single scripted HTTP request can inflate a malicious skill's perceived popularity by thousands. Their advice is blunt: treat every skill as untrusted code, because that's exactly what it is.

Scanning is equally fragile. The marketplace Orca studied runs security scans only at skill creation and at a popularity threshold, so the next scan "may be days or weeks later" — leaving a stale "clean" badge displayed while the underlying repository quietly changes. Worse, silent name-collision overwrites and blind bulk updates with no version pinning mean a skill that was benign yesterday can turn malicious on any update cycle.

Automated screeners don't fare much better. Unit 42 documented five malicious skills that passed VirusTotal and ClawScan between February and May 2026 — each one cleared by existing detection tools at the time of analysis. One used a 22 MB padding payload that simply exceeded scanner size thresholds, earning a clean verdict through sheer bulk.

The pattern across these failures:

• Popularity signals are spoofable with a single HTTP request
• Scans run only at creation or thresholds, not continuously
• Size limits and static rules let padded payloads sail through
• Unpinned updates can swap benign code for malicious code

Here's the important nuance: scanning itself isn't broken — static, threshold-bound scanning is. Snyk's tuned mcp-scan engine, combining deterministic rules with multi-model analysis, achieved 90–100% recall on confirmed malicious skills with 0% false positives on legitimate top-100 skills, per its audit of 3,984 skills. The lesson is that well-tuned, continuously updated detection works; one-shot checks at upload time do not.

That distinction shapes how SkillzBench is being built: every skill version pinned to a commit SHA, re-scanned when upstream changes, with security signals kept separate from usefulness scores rather than blended into a single misleading badge. Continuous verification isn't a luxury — it's the only approach the evidence actually supports.

The Assessment Landscape: Four Ways Security Teams Are Scoring Agent Trust

The security industry has not settled on a single way to decide whether an AI agent deserves trust — and the honest answer is that no one method covers everything. Four distinct approaches have emerged, each measuring something real and each leaving a different gap open.

The most formalized option comes from the standards track. An IETF Internet-Draft proposes 55 second-level metrics across four dimensions — Model-Native, Interaction, Operational and Basic Security — all expressed as pass rates so different agents can be compared directly. It also names threats older frameworks miss, including target hijacking, tool abuse, indirect prompt injection and memory poisoning. The catch: it remains a work in progress, valid for six months and explicitly not citable as reference material.

Domain benchmarks test capability rather than trustworthiness. CAIBench found models saturate on security knowledge at roughly 70% success but collapse to 20–40% in multi-step Attack & Defense scenarios. Its authors concede that pre-trained cybersecurity knowledge does not imply attack-and-defense ability — and that current benchmarks may not translate to real labor demands.

Commercial platforms take an operational view. A vendor landscape map segments the market into Agent-SPM, AI red teaming, MCP security, runtime defense, skill scanning and kill switches, noting that chatbot guardrails alone rarely suffice for production agents. Note the bias: the mapping vendor ranks itself first.

Skill-level trust scoring works from the artifact upward — pinning versions, scanning code, and scoring security, quality, license, maintenance and community signals. SkillzBench is building exactly this, alongside sandboxed benchmarking that reports Lift separately from ratings so capability never masquerades as safety.

• Standards benchmarks offer comparability but are immature and per-agent, not per-skill.
• Domain benchmarks reveal adversarial weakness but measure models, not the skills you install.
• Vendor platforms cover runtime and MCP risk, but evaluations stay isolated.
• Skill-level scoring reaches the artifact itself, yet still needs environmental context.

That last gap is the sharpest critique in the research: isolated evaluations miss environmental risk because "reach is a property of the environment." Roughly 1,280 third-party products now embed AI, only about 282 sit behind single sign-on, and around 1,000 stay invisible to identity infrastructure by default. Layered verification, not any single score, is where this is heading.

Layered Verification in Practice: What Good Skill-Level Trust Scoring Looks Like

Every trust score is only as good as the verification behind it — and the research shows most verification today has holes attackers already exploit. The question for any buyer isn't just "what's the score?" but "what does the scoring process actually defend against?"

Start with the documented failure modes. A demonstrated proof-of-concept showed install counts can be inflated with a single scripted HTTP request, and that marketplace scans run only at creation or at popularity thresholds — meaning a "clean" badge can be days or weeks stale while the underlying repository changes. Blind bulk updates with no version pinning let a benign skill turn malicious on any update cycle. Any evaluation framework worth trusting has to answer each of these directly.

Here's the checklist any buyer can apply, and how SkillzBench is building each pillar against a specific attack primitive:

• Version pinning to commit SHAs. Instead of trusting a package name or a blind update, every skill version is pinned to an exact commit — so what you evaluated is what you get, and silent name-collision overwrites can't swap the code underneath you.
• Continuous re-scanning. One-time badge scans go stale. Upstream changes are tracked, skills are re-scanned and re-benchmarked, and you're notified when something shifts — closing the gap that leaves old "clean" verdicts displayed over changed code.
• Capability shown separately from security. CAIBench research found models score around 70% on security knowledge but drop to 20–40% in multi-step attack-and-defense scenarios — knowledge doesn't equal adversarial capability. That's why SkillzBench publishes the Lift (performance with the skill versus without, in percentage points) apart from the trust score, never blended into one number.
• Per-agent loadouts. The sharpest critique of isolated assessment is that "reach is a property of the environment" — a skill's risk depends on what its agent can touch. SkillzBench's per-agent loadouts answer this directly: you toggle which skills and tools each agent sees, and the agent only sees what you chose.

The score itself is a 0–100 number built from five signal families: security, quality, license, maintenance and community health. Money never touches it — the platform is free, funded by ads around blog content, so there's no listing fee or placement tier to game.

No single assessment covers everything. The IETF's agent security benchmark draft — 55 metrics across four dimensions — is still a work in progress, and vendor platforms like red teaming and Agent-SPM operate at different layers. Snyk's audit of 3,984 skills found 13.4% carry at least one critical issue, which is why layered verification at the skill level matters now, before standards mature. The pillars above give you a concrete way to judge any trust score — including ours — on the defenses that documented attacks actually target.

Your Trust Checklist: Questions to Ask Before Installing Any Skill

So where does that leave you? With a checklist. None of these questions requires you to be a security engineer — they just require you to stop trusting by default.

Start with Orca Security's blunt advice: treat every skill as untrusted code, because that's exactly what it is. Their supply-chain research showed that install counts, badges, and "clean" scan verdicts can all be spoofed or go stale while a repository quietly changes underneath them. Popularity is not evidence of safety.

Before you install any skill, run it through this:

• Pin to a specific commit, never a mutable branch — Orca documented silent name-collision overwrites and blind bulk updates that let a benign skill turn malicious on any update cycle.
• Audit the publisher's provenance and do a line-by-line source review; Unit 42 found five malicious skills that passed VirusTotal and ClawScan screening between February and May 2026, including a 22 MB padding payload that earned a clean verdict by exceeding scanner size thresholds.
• Monitor outbound traffic against what the skill's documentation claims it does — unexpected network calls are your earliest warning of credential exfiltration.
• Treat any name collision as hostile until proven otherwise, and never accept a bulk update without re-reviewing what changed.
• Assess the agent itself on identity, permissions, connectivity, and activity — the framework security researchers propose for third-party agents, because "reach is a property of the environment," not the model.

The stakes justify the caution. Snyk's audit of 3,984 skills found 13.4% carried at least one CRITICAL issue and 36.82% had some security issue, and their researchers put it plainly: if you installed a skill in the past month, there's a 13% chance it contains a critical flaw. JPMorgan Chase's global CISO Patrick Opet argues agents should get "an identity but no entitlements by default" — the same principle applies to skills.

Doing all of this manually for every skill is exhausting, which is exactly why we're building SkillzBench: a free, independent directory that pins every version to a commit SHA, re-scans when upstream changes, and lets you give each agent a per-agent loadout so it only sees the skills you chose. You can browse verified skills, compare trust scores against the evidence behind them, and join the free waitlist at skillzbench.com to get early access to the directory and the MCP connector.