How to Build an Allowlist of Agent Skills for Your Org

Why Agent Skills Are an Ungoverned Attack Surface

In December 2025, security researchers at Cato CTRL documented a skill that looked like a harmless GIF Creator — until it silently downloaded and executed MedusaLocker ransomware. That same month, a state-sponsored campaign (GTG-1002) weaponized Claude Code, running 80–90% of its tactical operations autonomously against roughly 30 targets. Agent skills have become an attack surface most organizations don't even know they have.

The scale of the problem is measurable. A large-scale academic study of 31,132 skills found that 26.1% contain at least one vulnerability, while Snyk's independent audit of 3,984 skills put the figure at 36.82%, with 13.4% carrying a critical issue. Both studies point the same direction: roughly one in four to one in three skills is flawed.

What makes skills uniquely dangerous is how they inherit trust. Unlike a normal package, an installed skill loads repeatedly as trusted task-specific guidance — and it runs with the employee's full permissions: shell access, Git and cloud credentials, file systems, and messaging channels. Snyk researchers compare the ecosystem to the early days of npm and PyPI, "except with unprecedented access to credentials, file systems, and APIs."

Traditional security tooling barely sees any of this:

• Installing a skill is just markdown dropped into a user-writable directory — no installer, no admin rights, no package manifest.
• MDM tracks installed apps, not runtime-read text files.
• EDR stops at the process boundary; the skill's influence happens inside the agent's context.
• Marketplaces lack mandatory review — publishing requires only a SKILL.md file and a week-old GitHub account.

Human and model-level defenses don't fill the gap. Users approve 93–97% of Claude Code permission prompts, and vigilance drops from ~17% blocked early in a session to ~5% after 50+ prompts. A model's refusal is an opinion about what it should say; authorization is a decision about what your systems will do. Only code-enforced checks are reliable.

The risk compounds with complexity. Skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills, and automated frameworks like SkillJect hide payloads in auxiliary .py/.sh scripts while iteratively rewriting SKILL.md instructions until the payload executes. Meanwhile, daily skill submissions to public marketplaces jumped 10× in weeks — from under 50 to over 500 — with no vetting gate in sight.

This is the gap SkillzBench is being built to close: continuously discovering, security-scanning, and version-pinning skills so organizations can see which ones actually help and which ones carry risk. Until governance catches up, every unreviewed skill installed on an employee's machine is an approved agent taking actions nobody asked for.

Why Human Approval and Prompt Rules Don't Enforce Anything

Here's the uncomfortable math: users approve 93–97% of Claude Code permission prompts, according to enforcement research. Whatever a permission dialog was supposed to catch, in practice it catches almost nothing.

Vigilance degrades fast, too. Early in a session, humans block roughly 17% of dangerous commands; after 50+ prompts, that drops to about 5%. Across 1,053 paid testers, humans caught just 13.6% of dangerous commands while an automated classifier caught 89%. Approval fatigue isn't an edge case — it's the default operating condition of every team using agents daily.

Prompt-based rules fare no better. An analysis of 481 public CLAUDE.md files found that only 4.4%–16% of security rules had any matching built-in control. The rest were aspirations. A system prompt shapes what a model says, not what an orchestrator does — and as the repeat-after-me attack researchers put it, that gap is exactly where rejected skills still execute.

This distinction comes down to refusal versus authorization. A model's refusal is an opinion about what it should say; authorization is a decision about what your systems will actually do. As the same research frames it: "A refusal is a negotiation, and a patient attacker will keep negotiating. A permission check doesn't negotiate." Prompt injection remains unsolved by every major lab's own admission — OpenAI calls it unlikely to ever be fully solved, and NIST found all 13 frontier models fell to at least one attack across 250,000+ attempts. You cannot build a security control out of a mechanism the vendors themselves call unreliable.

The fix is enforcement at invocation time, in code:

• Pin every allowlisted skill to a reviewed version, so silent upstream changes trigger re-review rather than silently loading.
• Intercept the execution path, not just the installation — a denylisted skill that stays runnable is still a threat.
• Check every call against written policy, including arguments, and log decisions somewhere the agent cannot edit.
• Maintain two layers: inventory-level approval (what may load) and action-level approval (what an approved skill may do).

This is why SkillzBench treats allowlisting as a code-enforced control plane rather than a document: per-agent loadouts mean an agent only ever sees the skills and tools someone explicitly chose, with versions pinned and re-scanned on change. Given that large-scale research found 26.1% of skills carry at least one vulnerability, "trust the model to refuse" is not a policy — it's a hope. Approval governs inventory; interception governs behavior. Your allowlist needs to be the second thing.

The Four-Step Allowlist Playbook: Discover, Assess, Approve, Enforce

A skill installed on one laptop looks harmless — a markdown file in a user-writable directory. But that file inherits everything its owner can touch: shell access, credentials, cloud APIs, messaging channels. With large-scale research finding 26.1% of analyzed skills contain at least one vulnerability, orgs need a repeatable process, not ad-hoc judgment.

Step 1: Discover. Inventory every skill already installed across your fleet. Installation is a file-level change with no installer, no admin rights, and no package manifest, so traditional tooling misses it entirely — MDM sees apps, not runtime-read markdown. Treat AGENTS.md and CLAUDE.md rules files, hooks, and MCP servers as part of the same governed surface.

Step 2: Assess. Score each skill against five criteria:

• Capabilities — does it execute shell commands, fetch remote content, or stay instruction-only?
• Permissions — what does it inherit from the employee who installed it?
• Data reach — which files, repos, and services can it touch?
• Provenance — is the author verifiable, or a week-old GitHub account?
• Update behavior — can its content change silently after approval?

Weight script-bearing skills heavily: they are 2.12× more likely to contain vulnerabilities than instruction-only skills, and automated frameworks like SkillJect hide payloads in auxiliary .py/.sh files while rewriting SKILL.md until the payload executes.

Step 3: Approve a version-pinned baseline. Approve skills pinned to specific reviewed versions — never "latest." A decision matrix helps: approve read-only, known-author skills; restrict shell- or network-capable ones to defined teams; deny arbitrary command execution, credential access, or unknown origin. Because approved files sit in employee-editable locations, any change to a pinned version must trigger re-review.

Step 4: Enforce at invocation time. Approval at install time is not enforcement. As one analysis puts it, a refusal is a negotiation — and users approve 93–97% of permission prompts anyway, with vigilance collapsing from ~17% to ~5% after 50+ prompts. Check every skill invocation against written policy, including arguments, and log every decision somewhere the agent can't edit.

This is the workflow SkillzBench is being built to support — continuous discovery, version-pinned verification, and per-agent loadouts so each agent only sees the skills someone deliberately chose. Skills, proven: it starts with knowing exactly what's running.

Screening Criteria: What to Allow, Restrict, and Deny

Not every skill deserves a spot on your allowlist, and the data says the difference is measurable. In a large-scale analysis of over 31,000 skills, those bundling executable scripts were found to be 2.12× more likely to contain vulnerabilities than instruction-only skills. That single statistic should shape your default posture.

The safest baseline is to prefer instruction-only skills — SKILL.md files plus text references with no code to execute. This aligns with how SkillzBench is approaching its launch scope: instruction-only skills get full verification and benchmarking, while script-bearing skills are listed link-out only. When a skill ships with auxiliary .py or .sh files, it also becomes a delivery vehicle for stealth — the SkillJect research showed automated attacks hide payloads in those scripts while rewriting SKILL.md instructions until the payload executes.

Screen every candidate against the documented threat taxonomies. The academic study identified 14 vulnerability patterns spanning prompt injection, data exfiltration, privilege escalation, and supply chain risks, while Snyk's audit of ClawHub skills mapped eight policy categories: malicious code, suspicious downloads, credential handling, secret detection, third-party content exposure, unverifiable dependencies, and direct money access. A skill that trips any high-severity pattern — the study found 5.2% of skills exhibit patterns strongly suggesting malicious intent — should be denied outright.

A practical decision matrix looks like this:

• Allow read-only, narrowly scoped skills from known authors, pinned to a reviewed version.
• Restrict skills with shell or network access to specific teams with a documented need.
• Deny anything requesting arbitrary command execution, credential access, or direct money movement.
• Deny skills of unknown origin, or those that changed silently after approval.

Pinning is what makes the allowlist enforceable at all. Because skills live in employee-writable directories, every attribute you approved can change after the fact — so pin each skill to a specific commit SHA and trigger re-review whenever upstream changes. This is exactly the verify-and-monitor workflow SkillzBench is building: pin every version to a commit SHA, security-scan it, and re-scan when the upstream repository moves.

Finally, don't rely on the model or the human to catch what screening misses. Users approve 93–97% of Claude Code permission prompts, and vigilance collapses over long sessions. Automated vetting is demonstrably feasible — SkillScan achieved 86.7% precision and 82.5% recall, and Snyk's critical-level detectors hit 90–100% recall with zero false positives on curated legitimate skills. Screen with code, pin to a SHA, and re-review on change.

Scoping Agent Identity and Staying Current

An allowlist tells agents what skills they may load — but it says nothing about what an approved skill can do once loaded. Closing that gap requires two complementary controls: scoping each agent's identity and permissions, and continuously re-vetting skills as they change upstream.

The Hugging Face breach in July shows both sides of this equation. An autonomous AI agent intrusion executed roughly 17,600 attacker actions over about 4.5 days, and while real permission checks denied 84 operations against credential stores, a single shared over-scoped connector credential enabled cluster-admin access in under 13 hours. The lesson: per-agent identity with short-lived, job-scoped credentials is what turns a breach into a blocked attempt. The same failure mode appeared at PocketOS, where a Cursor agent deleted a production database volume in 9 seconds using a long-lived over-scoped token.

The recommended gate, per security analysis of agent authorization, has four parts:

• Every agent gets its own identity — never a shared service account.
• Credentials are short-lived and scoped to the specific job, not ambient and permanent.
• Every call is checked against written policy, including its arguments.
• Every decision is logged somewhere the agent itself cannot edit.

Identity scoping only works if the governed surface is complete. Skills don't operate alone: rules files like AGENTS.md and CLAUDE.md, hooks, and MCP servers all shape agent behavior and belong under the same allowlist discipline. Malicious instructions hidden in AGENTS.md have already been shown to trick OpenAI Codex into silently accessing AWS credentials, npm tokens, and Git configuration. The MCP ecosystem compounds the risk — researchers found over 1,800 MCP servers on the public internet running without authentication enabled, with tools often holding "ambient authority exceeding necessary privileges." Treating these artifacts as one inventory, pinned and reviewed together, prevents attackers from routing around your skill allowlist through a side door.

Finally, allowlisting is a snapshot of a moving target. Skill files sit in employee-editable locations, so every allowlist attribute — author, version, contents — can change after approval. That's why pinned versions must trigger re-review on change, and why continuous re-scanning matters more than a one-time audit. Snyk's audit of ClawHub found 36.82% of scanned skills carry at least one security flaw, and skill submissions grew 10× in weeks, meaning yesterday's clean skill can be today's compromised one.

This is the problem SkillzBench, AIQ Labs' independent testing lab for agent skills, is building for: pinning every skill version to a commit SHA, re-scanning and re-benchmarking as upstream skills change, and tracking changes so teams get notified rather than surprised. For organizations, the operational takeaway is simple — scope identity tightly, govern the whole surface, and never stop watching.