AI Supply Chain Attacks: The Next npm Incident Will Be a Skill

Skills Are the New Packages — With Weaker Defenses

The September 15, 2025 Shai-Hulud attack made history as the first successful self-propagating attack in the npm ecosystem, with compromised packages automatically harvesting tokens and publishing malicious versions of anything they could reach. CISA confirmed 500+ packages compromised and issued federal guidance on credential rotation and version pinning. Wiz's assessment of one payload being AI-generated signals that attackers are already using AI to author supply chain malware at scale.

Agent skills replicate this risk model — and in some ways amplify it. The first large-scale empirical study of agent skills analyzed 31,132 unique skills and found that 26.1% contain at least one vulnerability, with 5.2% showing high-severity patterns strongly suggesting malicious intent. Marketplaces aggregate skills from public repositories without mandatory security review, directly paralleling the npm/PyPI risk model.

Orca Security's research on a prominent skills marketplace documented four exploitable primitives that make skills even harder to defend than packages:

• Spoofable install counts via unauthenticated requests with no rate limiting, inflating popularity signals by several thousand
• Non-deterministic scanning — skills audited only at creation and popularity thresholds, leaving days-to-weeks detection windows
• Silent skill override, where same-name installs overwrite existing skills with no warning
• Blind bulk updates with no diffs or version pinning, enabling "delayed weaponization" attacks

The unique multiplier is autonomous execution combined with what researchers call the consent gap. Unlike npm packages, skills load dynamically based on task context, reducing human review opportunities. Once a skill is approved, it gains persistent permissions — file read/write, code download, network connections — without further prompts, corresponding to OWASP's Agentic Skills Top 10 (AST10) "Identity and Privilege Abuse" category.

NVIDIA's AI Red Team confirms that writing to skills files is a viable path to remote code execution even without shell access, and that LLM-as-a-judge guardrails are "predisposed to accept" commands like npm install — making them equivalent to arbitrary command execution when attacker-influenced. Their conclusion is blunt: prompt-based defenses are routinely subverted, and controls must be enforced outside of the model's control plane.

Real attacks have already happened. Orca's PoC skills achieved confirmed code execution on unique machines across different networks, proving this is not just theoretical risk. In December 2025, Cato CTRL researchers demonstrated a benign-appearing "GIF Creator" skill silently downloading and executing MedusaLocker ransomware, while the GTG-1002 campaign showed state-sponsored actors weaponizing Claude Code with malicious MCP servers — with 80–90% of tactical operations running autonomously.

This is exactly the trust gap that's throttling enterprise adoption. Cisco found 85% of enterprises have AI agent pilots underway but only 5% have reached production — an 80-point gap that Jeetu Patel frames as "a rational response to a genuine security problem," not skepticism about AI's potential.

SkillzBench is building the independent verification layer this ecosystem needs: continuous crawling and deduplication of skills from GitHub, every version pinned to a commit SHA, security scanning, license checks, and sandboxed benchmarking that publishes real Lift in percentage points. The launch scope deliberately favors instruction-only skills — script-bearing skills are 2.12× more likely to contain vulnerabilities, so they're listed link-out only.

Skills, proven. Join the SkillzBench waitlist at skillzbench.com to get independent security scans, commit-pinned versions, and real benchmark Lift for the skills your agents run.

The Attacks Have Already Started

The theory is already behind the curve. In December 2025, Cato CTRL researchers demonstrated a benign-looking "GIF Creator" skill that silently downloaded and executed MedusaLocker ransomware — a payload delivered not through a browser exploit or a phishing email, but through the instruction layer of an AI agent, according to the first large-scale empirical study of agent skills.

Orca Security went further, building end-to-end proof-of-concept attacks against a live skills marketplace that achieved confirmed code execution on real machines across different networks — "not just theoretical risk," as the researchers put it. Their technical writeup identified four exploitable primitives that any marketplace operator should lose sleep over:

• Spoofable install counts — unauthenticated GET requests with no rate limiting let Orca inflate counts to "several thousand," manufacturing social proof.
• Silent skill override — installing a same-named skill overwrites the existing one with no warning to the user.
• Blind bulk updates with no diffs and no version pinning, enabling "delayed weaponization" attacks.
• Non-deterministic scanning — skills are checked only at creation and popularity thresholds, leaving days-to-weeks detection windows.

The threat isn't confined to criminal actors. The same research documents the GTG-1002 campaign, in which state-sponsored actors weaponized Claude Code through malicious MCP servers, with 80–90% of tactical operations running autonomously. Nation-state tradecraft has arrived at the skill layer.

Against this backdrop, the enterprise numbers make sense. Cisco found that 85% of enterprises have AI agent pilots underway, but only 5% have reached production. Cisco's Jeetu Patel framed the gap bluntly: "With a chatbot, the worst case is a wrong answer. With an agent, the worst case is a wrong action, and some actions can't be undone." That 80-point gap isn't skepticism about AI's potential — it's a rational response to a genuine security problem.

The academic evidence backs the skeptics up: 26.1% of analyzed skills carry at least one vulnerability, and 5.2% show high-severity patterns strongly suggesting malicious intent. This is precisely the trust gap SkillzBench is being built to close — commit-pinned versions, continuous re-scanning on upstream changes, and per-agent loadouts that limit what a compromised skill can ever touch. Until verification like that exists at scale, the 85/5 gap will stay exactly where it is.

How to Defend: Package-Manager Rigor for Skills

If a worm can compromise 500+ npm packages overnight, a skills marketplace with no version pinning is a wider open door. The good news: the defense playbook already exists — it's the same rigor package managers, container registries, and CI/CD pipelines learned the hard way, applied to SKILL.md files.

Start with pinning skills to specific commit SHAs. When CISA responded to the Shai-Hulud npm worm, its federal guidance included pinning dependencies to known-safe releases (per the agency's alert — see the official CISA alert). Orca Security found the same gap in skills: blind bulk updates with no diffs or version pinning enabled delayed weaponization attacks. This is why SkillzBench is building verification that pins every skill version to a commit SHA.

Second, prefer instruction-only skills. The first large-scale empirical study of 31,132 unique skills found script-bearing skills are 2.12× more likely to contain vulnerabilities than instruction-only skills (OR=2.12, p<0.001), per the arXiv research. That finding — combined with NVIDIA's discovery that skills files are a file-write path to remote code execution — is why SkillzBench's launch scope covers instruction-only skills, with script-bearing skills listed link-out only.

Third, demand continuous re-scanning, not point-in-time audits. Orca showed that marketplaces scan skills only at creation and popularity thresholds, leaving days-to-weeks windows where malicious modifications display clean audit results. A skill that was safe when you installed it is not safe forever.

Fourth, enforce least-privilege per agent. The academic study identifies the "consent gap" — once approved, a skill gains persistent file, network, and download permissions with no further prompts — as the core systemic weakness. The fix is per-agent loadouts where the agent only sees the skills and tools explicitly chosen for it.

Fifth, keep controls outside the LLM's control plane:

• Prompt-based defenses are "routinely subverted" — controls must be enforced deterministically, outside the model, per NVIDIA's AI Red Team.
• Treat arbitrary code execution as the single highest-impact risk in an accessible agent.
• With sufficient time and expertise, nearly all frontier models can still be subverted — so don't rely on the model to police itself.

The bottom line from Orca's researchers: treat every skill as untrusted code, because that's exactly what it is. The empirical evidence backs them — 26.1% of skills carry at least one vulnerability, and 5.2% show high-severity patterns suggesting malicious intent. Package-manager rigor isn't optional for skills; it's the minimum bar.

Skills, proven. Join the SkillzBench waitlist at skillzbench.com to get independent security scans, commit-pinned versions, and real benchmark Lift for the skills your agents run.

What Independent Verification Should Look Like

The bar for trusting a skill should be no lower than the bar for trusting an npm package. Yet the first large-scale empirical study of agent skills found that 26.1% of skills contain at least one vulnerability, with 5.2% showing high-severity patterns suggesting malicious intent — a level of risk that demands a defined verification standard, not vibes.

That standard already has a shape. OWASP's Agentic Skills Top 10 (AST10) is the first comprehensive security framework for agent skills, and independent security researchers argue the same rigor applied to package managers, container registries, and CI/CD must now extend to skill marketplaces. In practice, that means every skill should pass through a consistent pipeline before it earns a place in your agent.

At minimum, independent verification should include:

• Security scanning — continuous, not point-in-time. Orca Security showed that scanning only at creation and popularity thresholds leaves days-to-weeks windows where malicious modifications appear clean.
• Commit-pinned versions — CISA's guidance after the Shai-Hulud npm worm, which compromised 500+ packages, was to pin to known-safe releases. Blind bulk updates with no diffs enabled "delayed weaponization" attacks in Orca's research.
• License checks and quality judging, so legal exposure and broken instructions surface before install, not after.
• Independent benchmarking — testing each skill in a sandbox with the same model, with and without the skill, and reporting the Lift in percentage points separately from any trust score.
• A preference for instruction-only skills. Script-bearing skills are 2.12× more likely to contain vulnerabilities, and NVIDIA's AI Red Team confirmed that file writes to skills paths can achieve code execution without shell access.

Popularity signals don't substitute for any of this. Orca inflated install counts into the thousands with scripted unauthenticated requests, and LLM-based guardrails are "predisposed to accept" commands like npm install, per NVIDIA — so the controls must live outside the model's control plane entirely.

This is the evaluation standard SkillzBench is being built around. Its Verify pillar pins every skill version to a commit SHA, runs license checks, security scans, and quality judging; its Benchmark pillar tests skills in a sandbox and publishes Lift separately from its 0–100 trust score, so security ratings and performance claims never contaminate each other. Money never touches scores, and script-bearing skills are listed link-out only at launch — a direct response to the empirical risk data.

The npm ecosystem taught us what happens when verification is optional: 500+ compromised packages, a CISA federal alert, and at least one AI-generated payload. Skills deserve better — and "trusted source" should mean verified, pinned, scanned, and independently benchmarked, not merely popular.

Skills, proven. Join the SkillzBench waitlist at skillzbench.com to get independent security scans, commit-pinned versions, and real benchmark Lift for the skills your agents run.

Your Skill Security Checklist Before the Next Incident

When CISA's guidance after the Shai-Hulud npm worm included pinning to pre-September 16, 2025 releases, it confirmed what skill ecosystems should already assume: version pinning is not optional. Agent skills deserve the same pre-incident hardening — before, not after, the next compromise.

Start with the mindset. Treat every installed skill as untrusted code, because that is exactly what it is. A large-scale empirical study found 26.1% of agent skills contain at least one vulnerability, and 5.2% show high-severity patterns suggesting malicious intent. Reading a skill's SKILL.md and referenced files line by line is the minimum bar before it ever touches an agent.

Next, apply these checks to every agent in your environment:

• Pin every skill to a specific commit and disable blind bulk updates. Orca Security demonstrated that unpinned, no-diff update flows enable "delayed weaponization," where a clean skill turns malicious after passing its audit.
• Strip script-bearing skills from agents unless absolutely necessary. Script-bearing skills are 2.12× more likely to contain vulnerabilities than instruction-only skills, and NVIDIA's AI Red Team confirmed that writing to skill files is a viable path to code execution.
• Give each agent an explicit loadout. The "consent gap" — persistent permissions granted at install with no further prompts — is the core systemic weakness, so enforce least privilege: the agent only sees the skills and tools you deliberately chose.
• Monitor upstream changes continuously. Orca found marketplaces that scan skills only at creation leave days-to-weeks windows where modified skills display clean audit results. Re-scan on every upstream change, not once at install.

Remember why prompt-based defenses don't count here. NVIDIA's red team found LLM-based guardrails are routinely subverted and "predisposed to accept" commands like npm install — controls must be enforced outside the model's control plane, in deterministic tooling and configuration.

Doing this manually across every agent, every skill, every upstream change is a full-time job. That's the gap SkillzBench is building for: independent security scans, versions pinned to commit SHAs, per-agent loadouts, continuous upstream monitoring, and benchmark Lift so you know which skills actually help — not just which ones are safe.

Skills, proven. Join the SkillzBench waitlist at skillzbench.com to get independent security scans, commit-pinned versions, and real benchmark Lift for the skills your agents run.