The Rise of the Agent Marketplace

The Skill Gold Rush Outpaced Its Trust Infrastructure

Forty-two thousand skills. Two marketplaces. Zero mandatory security review. That is the state of the agent skill ecosystem today, and it should sound familiar to anyone who watched PyPI, NPM, or early browser extension stores learn their security lessons the hard way.

A large academic study collected 42,447 skills from two community marketplaces and analyzed 31,132 unique ones after deduplication. The findings are stark: 26.1% of analyzed skills contain at least one vulnerability, with 13.3% exhibiting data exfiltration patterns and 11.8% privilege escalation. More troubling still, 5.2% show high-severity patterns the researchers say strongly suggest malicious intent.

The distribution model explains why. Registries like skills.rest and skillsmp.com aggregate third-party skills from public repositories without mandatory security vetting — a structure that directly mirrors the early, chaotic days of browser extension marketplaces. As Cloud Security Partners puts it, "We're repeating the story with AI skills and agents."

The risk profile is also uniquely worse than traditional package dependencies. Skills bundle instructions that shape agent behavior, and once a user approves one, it gains persistent permissions to read and write files, download code, and open network connections without further prompts. OWASP's Top 10 for Agentic Applications classifies this pattern as Identity and Privilege Abuse. Skills that bundle executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only ones.

Real attacks have already followed the theory:

• Orca Security demonstrated three end-to-end attack flows — bait-and-switch, nested skill injection, and delayed weaponization — achieving persistent code execution through skills that passed platform audits.
• Cato CTRL showed a benign "GIF Creator" skill silently downloading and executing MedusaLocker ransomware.
• The GTG-1002 espionage campaign weaponized Claude Code with malicious MCP servers, with 80–90% of tactical operations running autonomously.

The trust signals that should help users choose safely are themselves broken. Orca's research found install counts are trivially spoofable, scanning happens only at creation and at a popularity threshold — "which may be days or weeks later" — and bulk updates ship with no version pinning. The researchers' conclusion is blunt: there is an urgent need for capability-based permission systems and mandatory security vetting before this attack vector is further exploited.

This is the gap SkillzBench is being built to fill — pinning every skill version to a commit SHA, security-scanning, and benchmarking each skill in a sandbox with and without it installed, so Lift is measured rather than assumed. Until that verification layer exists at scale, the honest advice is Orca's: treat every skill as untrusted code, because that's exactly what it is.

The Documented Danger: One in Four Skills Is Vulnerable

If someone handed you a package for your codebase with a one-in-four chance of containing a vulnerability, you'd never install it. Yet that's effectively the odds in the agent skill ecosystem: a large-scale academic analysis of 31,132 skills collected from two marketplaces found that 26.1% carry at least one vulnerability — 13.3% enabling data exfiltration and 11.8% enabling privilege escalation.

A separate audit points the same direction. A Snyk-powered audit by Cloud Security Partners examined 3,984 skills and found 13.4% (534) contained at least one critical-level security issue, with 36.82% showing a flaw at some severity. The researchers' verdict is blunt: the ecosystem is repeating the security failures of PyPI, NPM, and RubyGems — only this time the "dependencies" are instructions that shape how your agent behaves.

The risk isn't evenly distributed. Skills that bundle executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills, a statistically significant finding from the same study. That's precisely why SkillzBench is building its launch scope around instruction-only skills, with script-bearing skills listed link-out only rather than vetted as installable packages.

And attacks are no longer theoretical. In December 2025, Cato CTRL showed a benign-looking "GIF Creator" skill could silently download and execute MedusaLocker ransomware. Orca Security went further, demonstrating three end-to-end attack flows — bait-and-switch, nested skill injection, and delayed weaponization via update — that achieved persistent code execution through skills that passed the platform's own security audits, with callback logs confirming real users were affected.

Part of what makes this exploitable is the consent gap. Users approve an entire skill without reviewing its capabilities; once approved, it gains persistent permission to read and write files, download code, and open network connections without further prompts. Orca also found the trust signals users rely on are gameable:

• Install counts can be spoofed, so popularity proves nothing about safety
• Skills are scanned only at creation and at a popularity threshold — which may be days or weeks later
• A name collision can silently override one skill with another
• Bulk updates ship with no version pinning, so what you installed may not be what you're running

The takeaway from both the academic and security communities is the same: treat every skill as untrusted code until it's been pinned, scanned, and measured. That's the gap SkillzBench is being built to close — version pinning to commit SHAs, continuous security scanning, and a 0–100 trust score where money never touches the rating.

Want the trust layer before you install another skill? Join the SkillzBench waitlist and get independent security scans and benchmarks as they ship.

CTA text: Join the waitlist — Skills, proven. Social proof: Independent testing lab by AIQ Labs. 26.1% of skills carry a vulnerability — know which ones before you install.

The Verification Layer Emerges — and Lift Becomes the Metric That Matters

The market's answer to the trust vacuum is taking shape: a formal verification layer, built by some of the biggest names in AI infrastructure. NVIDIA now runs a three-tier pipeline — safety scanning, distinctiveness checks, and live sandboxed evaluation — over 300+ verified skills, reporting an average Skill Lift of 41 points, with Correctness jumping from 46 to 87 with the skill active, according to NVIDIA's SkillEvaluator writeup. OpenAI and MLflow have published formal evaluation methodologies of their own, converging on the same principle: skills are software components, and software gets tested.

The case for testing is not abstract. Skills are startlingly fragile. MLflow engineers watched a Correct Tool Selection score swing from 43% to 98% after a single instruction change. A one-word shift from "optional" to "mandatory" in a config comment broke more than 90% of test cases, per Langfuse's evaluation study. If a single comment can move outcomes that far, star ratings and vibes-based reviews tell you almost nothing.

Just as important: skills can hurt, not just help. NVIDIA measured per-product Skill Lift ranging from +2 to +46 points, and token effects cut both ways — one skill reduced token usage by 76.9%, another inflated it by 120.3%. A skill that loads when it isn't relevant can drag an agent down. The only honest way to know a skill's value is to run the same agent, on the same model, in a sandbox, with and without the skill — and publish the difference.

That measured difference is what the industry is starting to call Lift, and it's becoming the metric that matters. OpenClaw is piloting an Evals tab at the point of adoption, and Nous Research added a pre-install security scan. The direction is unmistakable: versioning, scanning, benchmarking, and trust scoring — the full software-supply-chain treatment.

This is exactly the gap SkillzBench is being built to fill. Its benchmarking pillar tests every instruction-only skill in a sandbox under identical conditions, with and without the skill enabled, and reports Lift in percentage points — always shown separately from its 0–100 trust score, with money never touching either. Because skills bundling executable scripts are 2.12× more likely to contain vulnerabilities, per the large-scale arXiv study, SkillzBench's launch scope focuses on instruction-only SKILL.md packages, listing script-bearing skills link-out only.

The takeaway for anyone adopting skills today: never assume a skill helps just because it exists. Treat it like any untested dependency — measure it, or use a source that does. If you want Lift scores and security signals you can actually trust, join the SkillzBench waitlist at skillzbench.com and see the numbers before you install.

What a Trustworthy Skill Supply Chain Looks Like in Practice

Knowing that 26.1% of analyzed skills carry at least one vulnerability, the question shifts from "should we be careful?" to "what does careful actually look like?" The researchers behind the largest skill security study to date call for "capability-based permission systems and mandatory security vetting" — but that abstract demand needs concrete practices teams can run today.

Start with version pinning to commit SHAs. Orca Security's investigation of a prominent marketplace found blind bulk updates with no version pinning, enabling delayed weaponization attacks that passed platform audits. Pinning every skill to a specific commit closes that door. Scanning must also be continuous, not a one-time gate: Orca found skills were scanned only at creation and again at a popularity threshold, "which may be days or weeks later" — leaving a wide window for malicious updates.

Then there's the question of what you install at all. Skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only packages, which argues for avoiding script-bearing skills by default. Benchmarking requires equal discipline: because small wording changes swing outcomes dramatically — MLflow saw correct tool selection jump from 43% to 98% after a single instruction edit — skills must be tested with the same model, with and without the skill, at a pinned version.

In practice, a trustworthy supply chain looks like this:

• Pin versions to commit SHAs so updates can never silently replace what you approved
• Scan continuously — re-scan on every upstream change, not just at first publish
• Default to instruction-only skills; treat script-bearing packages as link-out references, not installs
• Benchmark with the same model, with and without the skill, and track upstream diffs that could shift behavior

This is the exact blueprint SkillzBench is being built around: Discover (crawl and dedupe skills from GitHub), Verify (SHA-pinned versions with security, license, and quality checks), Benchmark (sandboxed with/without testing published as Lift), Equip (a single MCP server with per-agent loadouts), and Stay current (re-scan and re-benchmark on upstream changes). Two design choices matter most: Lift is always shown separately from trust scores, and money never touches ratings — because install counts and popularity signals are, as Orca demonstrated, trivially fakeable. Skills, proven — that's the standard the ecosystem now needs.

How to Adopt Skills Safely Today (and What's Coming)

You don't need to wait for the ecosystem to mature to adopt skills responsibly. The same research that documents the risks — a large-scale analysis finding 26.1% of 31,132 skills carry at least one vulnerability — also points to defenses you can put in place today. The principle is simple: treat every skill as untrusted code, because that's exactly what it is.

Start by reviewing skills before enabling them, the way you would any third-party dependency. Orca Security's advice is blunt: pin skill versions to specific commits and audit before installing, because marketplace scanning is non-deterministic — some skills are only re-scanned at a popularity threshold, which "may be days or weeks later" (Orca Security). Pinning to a commit SHA means the code you reviewed is the code that runs.

Second, restrict what each agent can see. The arXiv researchers call for capability-based permission systems because approved skills gain persistent access to files, network connections, and code execution without further prompts. A minimal per-agent policy — only the skills and tools relevant to the task — shrinks the blast radius when something goes wrong.

Third, demand measured Lift before adopting anything. NVIDIA's SkillEvaluator found per-product Skill Lift ranging from +2 to +46 points, and one skill increased token usage by 120.3% while another cut it 76.9% (NVIDIA's benchmarking work). A skill that loads when irrelevant can actively reduce agent performance, so run your own with-and-without tests rather than trusting install counts — which are trivially fakeable.

Your working checklist:

• Treat every skill as untrusted code; read the SKILL.md and any bundled files before enabling
• Pin each skill version to a specific commit SHA; re-review on any update
• Restrict per-agent skill and tool visibility so agents only see what their task requires
• Be extra cautious with script-bearing skills — they're 2.12× more likely to contain vulnerabilities
• Measure Lift with and without each skill before standardizing on it

The industry is converging on the same answer: skills should be versioned, scanned, benchmarked, and continuously monitored — software components, not vibes. SkillzBench is being built to make that operational: a control panel that pins every skill to a commit SHA, security-scans and license-checks it, benchmarks it in a sandbox with published Lift, and lets you toggle exactly which skills and tools each agent sees through a single MCP server. It's pre-launch, but teams that want versioned, scanned, benchmarked skills — Skills, proven — can join the waitlist at skillzbench.com today.