The Skills Explosion Nobody Can Verify

Your AI agent just installed a new skill. It looked helpful, it had good reviews, and it now has access to your shell, your files, and your credentials. There is no version pinning, no inspection tool, and no registry review standing between that skill and your environment — and the numbers say you should be worried.

A large-scale academic study analyzed 31,132 unique skills across two marketplaces and found that 26.1% contain at least one vulnerability, spanning prompt injection, data exfiltration, privilege escalation, and supply chain risks. Separately, Snyk's ToxicSkills audit of 3,984 skills found 13.4% carry at least one critical security issue and 36.82% have at least one flaw of any severity — including 76 confirmed malicious payloads. These are different studies with different corpora, but they converge on the same conclusion: roughly one in four to one in three skills is not safe to run as-is.

The trust infrastructure to catch these simply does not exist. As Mondoo's analysis documents, the ecosystem has:

• No version pinning — a skill you approved can be silently updated with malicious code
• No inspection tooling — nothing equivalent to npm audit for skills
• No mandatory registry security review before publication
• No organizational visibility into which skills are installed where

This creates what Mondoo calls the core paradox: every platform tells users to "only run trusted skills," but that assumes users can verify what a skill does before running it. Most can't. A compromised npm package runs code; a compromised skill runs code inside an agent that already holds your credentials.

Meanwhile, attackers are following the growth. Daily skill submissions jumped from under 50 to over 500 in a matter of weeks — a 10x spike that Snyk describes as an ecosystem at an inflection point. The ClawHavoc campaign flooded a major registry with 341 malicious skills in just three days, all sharing a single command-and-control server.

And popularity offers no protection. A code-review skill with 312 GitHub stars was found to contain hidden HTML-comment instructions that auto-approve "security-exempt" code and exfiltrate conversation context. As Snyk put it: the skill author did nothing wrong, the user installed a popular, well-reviewed skill — yet the agent was compromised. Stars measure attention, not trust.

This is the gap SkillzBench is being built to close: version-pinned verification, security scanning, and independent benchmarking for a layer of the AI stack that currently has neither.

Why Skills Are a New Class of Supply Chain Risk

A compromised npm package runs code. A compromised skill runs code inside an AI agent that already has your credentials, your shell, and your environment — and that difference is why the npm playbook doesn't transfer. As Mondoo's analysis puts it, the blast radius is simply larger.

Three properties make skills a genuinely new class of supply chain risk rather than a repackaged one:

• Full permission inheritance. A skill doesn't request its own permissions — it operates with everything the agent already has: shell access, filesystem access, credentials, and persistent memory.
• Progressive loading. Skills load only ~100 tokens at startup, so malicious payloads can hide in scripts and references that load later, after the user has already granted trust.
• Instructions as attack surface. Malicious intent can live in natural-language SKILL.md text that no scanner flags as dangerous code.

That third property is the most unsettling. Duke University researchers demonstrated a threat class they call "dynamic malicious skills," where instructions embedded in SKILL.md documentation induce the agent to inject malicious logic into otherwise benign code at runtime — and their paper concludes that static code analysis is ineffective, because the original code contains no explicit malicious intent. The payload is the prose.

The numbers confirm that "just scan the code" is not enough. A large-scale academic study of 31,132 unique skills found that skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills. Meanwhile, CrowdStrike's taxonomy now documents over 200 distinct prompt injection techniques, including dormant "sleeping" instructions that look harmless during review and activate on a trigger later.

These findings set hard design constraints for any trust score. It must evaluate instructions and code together, not code alone. It must account for how a skill loads, not just what it contains. And it must treat script-bearing skills as a categorically different risk tier — which is why SkillzBench's launch scope fully verifies instruction-only skills while listing script-bearing skills link-out only, pending deeper runtime testing.

A trust score built on static scanning alone would miss exactly the attacks that matter most. The threat model demands more.

Anatomy of a Trust Score: Five Signals, One Number

Every platform tells you to "only run trusted skills" — but as Mondoo's analysis points out, that assumes you can verify what a skill does before running it. Most people can't. A trust score exists to close that gap: one number, built from signals you can actually inspect.

The design maps directly onto documented failures. A large-scale academic study found 26.1% of 31,132 analyzed skills contain at least one vulnerability, spanning prompt injection, data exfiltration, privilege escalation, and supply chain risks. Snyk's separate audit of 3,984 skills found 13.4% carry a critical issue, including 76 confirmed malicious payloads. Any credible score must weigh five distinct signal families:

• Security — scan results across the 14 vulnerability patterns researchers have already catalogued, from injection to credential theft.
• Quality — whether the skill follows documented authoring best practices, like keeping SKILL.md bodies lean and testing across target models.
• License — clear, permissive terms, since skills are third-party code you're injecting into a credentialed environment.
• Maintenance — version pinning and upstream tracking, because skills can be silently updated with malicious code.
• Community — provenance signals that flag templated impersonation campaigns, like the actor behind 54% of confirmed malicious skills.

Crucially, the score can't rely on static scanning alone. Duke researchers demonstrated "dynamic malicious skills" where harmful logic hides in natural-language documentation and evades static analysis entirely — the original code contains no explicit malicious intent. That's why a score must combine deterministic scanning with LLM-assisted intent analysis, which can read instructions and code together "and reason about intent in a way regex never could." This mirrors the validated SkillScan approach, which achieved 86.7% precision and 82.5% recall — and it's the architecture SkillzBench is building into its Verify pillar.

Two boundaries keep the score honest. First, script-bearing skills — which are 2.12× more likely to contain vulnerabilities than instruction-only ones — get listed link-out only at launch, while instruction-only skills receive full verification. Second, benchmark Lift is always shown separately from the trust score. Popularity is not a trust signal: a code review skill with 312 GitHub stars contained hidden instructions to exfiltrate conversation context. A 0–100 score measures risk and reliability; Lift measures whether the skill actually helps. Conflating them lets stars mask danger — and money never touches either number.

From Score to System: How SkillzBench Is Building the Trust Layer

A trust score is only useful if it lives inside a system that keeps it honest over time. The research is blunt about why: as Mondoo documents, "nothing like this exists at registry level yet" — no version pinning, no standard inspection tooling, no mandatory security review, and no organizational visibility into what agents have installed.

SkillzBench is building that missing layer around five pillars. Discover continuously crawls and deduplicates skills from GitHub, because the ecosystem has grown past 50,000 skills across just two community registries with no central vetting. Verify pins every skill version to a commit SHA — directly answering the documented risk of skills being silently updated with malicious code — and combines deterministic security scanning, license checks, and LLM-assisted quality judgment, since Duke University researchers showed that static analysis alone misses malicious intent hidden in natural-language documentation.

The remaining pillars turn a score into ongoing infrastructure:

• Benchmark — every skill is tested in a sandbox with the same model, with and without the skill, and the resulting Lift is published in percentage points. Lift stays separate from the trust score, because popularity isn't a trust signal: a code-review skill with 312 GitHub stars was found hiding exfiltration instructions.
• Equip — a single MCP server delivers per-agent loadouts. You toggle skills and tools per agent, and the agent sees only what was chosen — least privilege in practice, as CISA/NSA interagency guidance recommends.
• Stay current — upstream changes are tracked, re-scanned, and re-benchmarked, with notifications when a pinned skill shifts underneath you.

The launch scope reflects the evidence, not convenience. Script-bearing skills are 2.12× more likely to contain vulnerabilities than instruction-only skills, per the large-scale academic study, so SkillzBench fully verifies instruction-only skills (SKILL.md plus text references) and lists script-bearing skills link-out only. Organizational allowlists and audit trails for teams are on the roadmap, mirroring the trusted-registry model agencies are already recommending.

Money never touches scores — trust comes from security, quality, license, maintenance, and community signals, nothing else. SkillzBench is pre-launch and actively being built. The waitlist is open for teams who want skills, proven.