Third-Party Skills Are the New Browser Extensions—With Worse Reviews

The 98,000-Skill Gold Rush Nobody Is Vetting

Ninety-eight thousand skills in ninety days—and nobody checked a single one before it went live. That's the scale of the third-party skill ecosystem right now, and it's the exact setup that made early browser extension marketplaces a security nightmare, only faster.

Two independent academic teams make the browser-extension parallel explicitly. The first large-scale security analysis of agent skills found that community registries "aggregate third-party skills without mandatory security review," mirroring early extension marketplaces where "rapid proliferation" left "security considerations secondary to functionality." A second study reaches the same verdict: registries indexed over 98,000 skills within three months of launch, "typically without security review—resembling early extension marketplaces with low friction and limited gatekeeping."

The numbers behind that gatekeeping vacuum are sobering. Of 31,132 skills analyzed in the first study, 26.1% carry at least one vulnerability, spanning prompt injection, data exfiltration, privilege escalation, and supply-chain risks. The breakdown is specific:

• 13.3% of skills exhibit data exfiltration patterns
• 11.8% show privilege escalation behaviors
• 5.2% contain high-severity patterns "strongly suggesting malicious intent"
• Skills bundling executable scripts are 2.12× more likely to be vulnerable than instruction-only skills

The second study adds behavioral ground truth: from 98,380 skills analyzed, 157 were confirmed malicious with 99.6% precision, tied together by 13 attack techniques across six kill chain phases. A single actor accounted for 54.1% of confirmed cases through templated brand impersonation—this isn't hobbyist malware, it's industrialized attack production.

Here's the core problem for anyone installing skills today: there is no way to tell a helpful skill from a harmful one before you install it. The registry listing tells you what a skill claims to do. It tells you nothing about what the SKILL.md instructions actually instruct your agent to do, whether the bundled scripts exfiltrate credentials, or whether a "sleeping" instruction will activate weeks after review passes.

Even Anthropic's own documentation concedes the gap, stating that skills "can introduce vulnerabilities" and advising users to "thoroughly audit untrusted skills"—while positioning its automated reviews as a complement to, not a replacement for, manual scrutiny.

This is the void SkillzBench is being built to fill: independent verification, version-pinned security scanning, and sandboxed with/without benchmarking, so a skill's actual Lift and risk are visible before it ever touches your agent. Until infrastructure like that exists at scale, every install is a trust decision made blind.

Why Skills Are More Dangerous Than Extensions Ever Were

Browser extensions at least asked before they acted. Skills grant themselves full local user privileges the moment you install them, and then never ask again.

That's what behavioral security research identifies as the core escalation: installing a skill "typically grants it full local user privileges, with minimal scrutiny or interactive confirmation." Once approved, it can read and write files, download code, and open network connections without further prompts. OWASP classifies this "consent gap" as Identity and Privilege Abuse—and it eliminates the exploitation stage attackers traditionally needed.

The second escalation is structural. Skills combine the code-execution privileges of native extensions with the prompt-injection attack surface of natural language. A malicious SKILL.md doesn't need to exploit a memory corruption bug; the instructions themselves are the payload. As one practitioner bluntly put it, a malicious skill is "a prompt injection you installed yourself" (Jannik Reinhard).

The data quantifies the risk gradient. According to the first large-scale empirical study of agent skills, skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills (p<0.001). That finding directly shapes how SkillzBench is scoping its launch: instruction-only skills get full validation, scanning, and sandboxed benchmarking, while script-bearing skills are listed link-out only.

Real attacks prove the threat isn't theoretical:

• A "GIF Creator" skill demonstrated delivering MedusaLocker ransomware through an agent workflow (Cato CTRL, December 2025).
• The GTG-1002 espionage campaign weaponized Claude Code with malicious MCP servers, with 80–90% of tactical operations running autonomously.
• A single actor accounted for 54.1% of confirmed malicious skills via templated brand impersonation—attack production is industrialized.

Here's why one-time review keeps failing: attackers are building for it. CrowdStrike's taxonomy now documents over 200 distinct prompt-injection techniques, including "Trigger-Activated Rule Addition"—sleeping instructions that activate only later, appearing harmless during review—and "Algorithmic Payload Decomposition," which fragments malicious instructions to evade scanners. The behavioral study confirms the pattern: "shadow features" (undocumented capabilities) appear in 100% of advanced attacks but 0% of basic ones. Sophisticated attackers know exactly what a static review will see.

This is the structural argument for continuous verification rather than install-time trust. SkillzBench's approach—pinning every version to a commit SHA, then tracking upstream changes to re-scan and re-benchmark as skills evolve—exists precisely because the skill you audited last month may not be the skill running on your machine today. Responsible disclosure got 93.6% of confirmed malicious skills removed within 30 days, but removal happens only after someone detects the problem. In an ecosystem where 5.2% of skills exhibit high-severity patterns strongly suggesting malicious intent, detection needs to be continuous, not a one-time gate at the registry door.

Security Scores Aren't Enough: Skills Can Hurt Performance Too

A skill can pass every security scan and still make your agent worse at its job. That's the uncomfortable finding buried in NVIDIA's benchmark data, and it explains why "is it safe?" and "does it help?" are two questions the current ecosystem answers with a shrug.

NVIDIA's SkillEvaluator, run across 300+ verified skills and 30+ products, measured an average Skill Lift of +41 points for Correctness (46→87) and +39 for Effectiveness (39→78). On the surface, that looks like a ringing endorsement of the skill ecosystem. But averages conceal the per-skill chaos underneath.

The same benchmark documented token-usage outcomes ranging from a 76.9% reduction (the jetson-optimize-memory skill cut tokens from 617,306 to 142,540) to a 120.3% increase (cuopt-install, which also added 20.8% execution time and was flagged as needing optimization). A skill that doubles your token spend while shaving correctness is not a neutral addition—it's a tax you installed yourself.

NVIDIA's own analysis explains why: every skill in an environment competes for the agent's attention, and a skill that loads when it's not relevant actively degrades performance. Per-product Skill Lift ranged from +2 to +46 points, meaning the same skill can be a game-changer in one harness and dead weight in another. What to watch for:

• Token inflation—skills that balloon context windows and cost per task
• Negative interference—irrelevant skills loaded at startup diluting attention
• Harness-dependent results—a +46-point skill in one agent may be near-useless in another
• Execution-time drag—overhead that shows up only under real workloads

This is the "worse reviews" part of the browser-extension parallel. Chrome extensions at least accumulated user ratings over years of install history; skill registries index tens of thousands of packages with no performance signal at all. A README tells you what a skill claims to do. Nothing currently tells you what it actually does to your agent's output, cost, or latency.

The security research is starting to mature—static analysis, behavioral verification, government guidance—but efficacy measurement remains nearly untouched. Emerging approaches like NVIDIA's with/without A/B evaluation and OpenClaw's Evals tab point in the right direction, but they're isolated pockets, not infrastructure.

This gap is exactly why SkillzBench is being built around the principle that Lift is always shown separately from security ratings: benchmarking every skill in a sandbox with the same model, with and without the skill, so the "does it help?" question gets a number instead of a guess. As NVIDIA puts it, a skill can only be measured as precisely as its evaluation set describes the job—and until the ecosystem measures it at all, buyers are browsing a marketplace where nothing has been road-tested.

What Trusted Skill Verification Looks Like

If a quarter of everything in a store were defective, you wouldn't shop there—you'd demand a certification program. That's exactly where the agent-skills ecosystem is heading, and the emerging answer looks a lot like what governments and researchers have already prescribed.

The most authoritative blueprint comes from government itself. CISA/NSA guidance, issued with Five Eyes partners, recommends maintaining "a registry of trusted third-party components" and restricting agents to approved lists only. In other words: the trusted-registry model isn't a vendor pitch—it's official best practice for exactly the problem skills create.

Static scanning alone, however, isn't enough. The study that behaviorally analyzed 98,380 skills achieved 99.6% precision by actually observing what skills do—a 150× improvement over static-only approaches. This matters because static flagging overstates confirmed maliciousness: 26.1% of skills showed vulnerabilities in static analysis, but only 157 were behaviorally confirmed as malicious. Verification needs both layers.

The second pillar of the emerging standard is with/without benchmarking. NVIDIA's SkillEvaluator work measured skills by running identical tasks with and without each skill installed, and OpenClaw now publishes these results directly on skill listings. The reason is simple: per-skill outcomes swing wildly, from one skill cutting token usage by 76.9% to another inflating it by 120.3%. A security score can't tell you that.

This is the gap SkillzBench is being built to fill, pre-launch, around five pillars:

• Version pinning — every skill versioned to a specific commit SHA, so what was scanned is exactly what runs
• Sandbox benchmarking with the same model, with and without the skill, publishing the Lift in percentage points
• A 0–100 trust score built from security, quality, license, maintenance, and community signals—with Lift always shown separately, and money never touching scores
• Continuous re-scanning, because "shadow features" appeared in 100% of advanced attacks and delayed-trigger techniques look harmless during initial review

The scoping decisions follow the evidence directly. Since script-bearing skills are 2.12× more likely to contain vulnerabilities than instruction-only ones, SkillzBench launches with instruction-only skills fully tested, while script-bearing skills are listed link-out only. It's a conservative posture for an ecosystem where researchers explicitly call for "mandatory security vetting" before the attack vector is further exploited.

Independent verification isn't a nice-to-have layered on top of the skills ecosystem. It's the infrastructure the ecosystem needs before it can be trusted at all.

How to Vet Skills Before You Install Anything Today

Until independent verification infrastructure matures, your own review process is the only gate between a random SKILL.md and your machine. The stakes are real: installing a skill typically grants it full local user privileges with minimal scrutiny, and once approved, it can read files, download code, and open network connections without further prompts. Practitioner guidance puts it bluntly: treat installing a skill like installing software—and read the SKILL.md and scripts before you install.

That manual review matters more than it might seem. Skills that bundle executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills, according to an analysis of 31,132 skills that found 26.1% carry at least one vulnerability. Anthropic's own documentation echoes this, advising users to install only from trusted sources and thoroughly audit anything untrusted—while noting that automated security reviews should complement, not replace, manual code review.

Before installing anything today, work through a consistent checklist:

• Read the full SKILL.md file, not just the description. Look for instructions that direct the agent to fetch external content, disable safeguards, or act outside the stated task.
• Inspect every bundled script line by line. Check for network calls, file writes outside expected paths, and obfuscated or encoded strings.
• Prefer instruction-only skills. They carry roughly half the vulnerability risk of script-bearing ones and eliminate code-execution exposure entirely.
• Check the source repo's history, license, and maintainer. One actor accounted for 54.1% of confirmed malicious skills via templated brand impersonation, so a polished README proves nothing.
• Re-review after updates. "Shadow features"—undocumented capabilities—appear in 100% of advanced attacks, and CrowdStrike has documented delayed-trigger injections that look harmless during initial review.

For teams, the bar should be higher than individual diligence. CISA/NSA guidance from Five Eyes partners recommends maintaining a registry of trusted third-party components and using nothing off the approved list—an allowlist model that turns one-off vetting into an auditable policy.

Manual vetting doesn't scale, though, and it can't tell you whether a skill actually helps. NVIDIA's benchmarking found skills swing performance wildly—one cut token usage by 76.9% while another increased it by 120.3%—so a safe skill can still be a bad one. That's the gap SkillzBench is being built to close: continuous scanning, version pinning, sandboxed before/after benchmarking, and trust scores with performance Lift shown separately. Until it launches, you can join the waitlist to get verified, benchmarked skills the moment they're available—and keep reading every SKILL.md until then.