How to Audit Third-Party Agent Skills Before Using Them
By SkillzBench · October 15, 2026
Learn a 15-minute audit workflow for third-party AI agent skills: spot malicious SKILL.md patterns, pin versions, and avoid rug pulls before installing.
How to Audit Third-Party Agent Skills Before Using Them
Key Facts
- 26.1% of 31,132 analyzed agent skills contain at least one vulnerability, a large-scale academic study found.
- Skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only ones, research shows.
- Snyk's ToxicSkills audit found 36.82% of 3,984 scanned skills had a security issue and 13.4% had a critical one, the report states.
- Mitiga demonstrated full codebase exfiltration using just 4 user interactions with zero logs, according to their proof of concept.
- 73% of tested AI platforms fail to adequately enforce at least one trust boundary, a systematization of prompt injection research found.
- Agentic AI CVEs grew 255% from 2024 to 2025, the largest jump of any AI subcategory, Trend Micro reports.
- Combining static analysis with LLM-based review achieves 86.7% precision and 82.5% recall detecting malicious skills, per the SkillScan framework.
Why Agent Skills Are an Unvetted Attack Surface
Your AI coding agent already has shell access, your credentials, and your file system. Every third-party skill you install inherits all of it — and the bar for publishing one is a Markdown file and a GitHub account that's a week old.
That asymmetry is the core problem. A Snyk audit found that publishing on ClawHub requires no code signing, no security review, and no default sandbox, while skills gain access to environment variables, messaging channels like email and Slack, and persistent memory across sessions. Meanwhile, the ecosystem is scaling faster than anyone can police it: daily skill submissions jumped from under 50 in mid-January 2026 to over 500 by early February — a 10x increase in weeks.
The numbers confirm the risk is not hypothetical. A large-scale study of 31,132 skills found 26.1% contain at least one vulnerability, with 5.2% exhibiting high-severity patterns that strongly suggest malicious intent. Snyk's independent scan of 3,984 skills found 36.82% had at least one security issue, and 13.4% contained a critical one. The threat moved from theory to reality in February 2026, when a coordinated malware campaign pushed 30+ malicious skills through ClawHub targeting Claude Code users.
The signals you'd normally rely on don't hold up here:
- Stars and upvotes can be manufactured — Mitiga demonstrated bot upvotes, fake downloads, and fake comments inflating trust on demand.
- A skill's full codebase exfiltration PoC required only 4 user interactions and produced no logs at all.
- Skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only skills.
- 73% of tested platforms fail to adequately enforce at least one trust boundary.
Worst of all, one-time approval is not a safety model. Rug-pull attacks deliberately modify skill behavior after it's been vetted and installed, and curl | source patterns let remote content change what a skill does after publication. A skill that was clean when you reviewed it can behave differently tomorrow.
This is why SkillzBench treats verification as a continuous process rather than a gate — pinning every skill version to a commit SHA, re-scanning on upstream changes, and re-benchmarking so a skill can't quietly become something you never approved. The audit you ran last week is only as good as the version it covered.
What to Screen For: The Threat Patterns and Red Flags
A quarter of the agent skills on public marketplaces can hurt you — and the malicious ones rarely look malicious. Large-scale academic analysis of 31,132 skills found 26.1% carry at least one vulnerability, spanning 14 distinct patterns across four categories: prompt injection, data exfiltration, privilege escalation, and supply chain risk. Your audit checklist should map directly to those categories.
Beyond the four categories, watch for skill-specific attack techniques documented in a systematization of prompt injection research: Tool Poisoning (malicious tool descriptions), Shadowing (context contamination), Tool Squatting (name-similar malicious tools), and Rules File Backdoors (hidden instructions in rules files). Rug Pull attacks are especially insidious — the skill behaves until after approval, then changes.
The hands-on red flags are concrete and greppable. Mitiga's researchers treat any silencing or allowlist instruction as an immediate red flag — an attacker co-designed a stealthier exfiltration attack with the agent itself in their proof of concept, which produced no logs across four user interactions.
- Obfuscated commands — base64 or Unicode-encoded payloads, often exfiltrating files like AWS credentials, per Snyk's ToxicSkills audit
- Unverifiable dependencies — any `curl | source` pattern lets attackers modify skill behavior after publication
- Password-protected ZIP downloads, a common external malware distribution vector
- Configuration findings like MCP auto-approval, YOLO mode enabled, or sandbox disabled
Snyk's ToxicSkills audit confirmed that 100% of malicious skills contain malicious code patterns, while 91% also use prompt injection to prime the agent into bypassing safety mechanisms. The two attack styles converge, so your review must cover both instruction text and any bundled scripts.
Adopt a source-sink lens, as OpenAI recommends: identify what untrusted content a skill ingests (sources) and what dangerous capabilities it can reach (sinks — data transmission, link-following, tool use). Dangerous actions should never happen silently.
Two structural facts should shape your prioritization. Skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only ones — which is why SkillzBench is scoping its launch benchmarks to instruction-only skills and listing script-bearing skills link-out only. And for detection, the validated approach combines static analysis with LLM-based semantic review: the SkillScan framework achieved 86.7% precision and 82.5% recall. Static scanning alone misses business-logic flaws; semantic review catches intent.
Skills, proven — join the SkillzBench waitlist at skillzbench.com to get early access to independent skill security scanning and benchmarking as we build it.
A Practical Pre-Use Audit Workflow
A structured audit takes 15–30 minutes per skill and dramatically reduces your exposure. The research consensus is clear: skills "execute with implicit trust and minimal vetting," so the burden of verification falls on you before installation, not after.
Here is a practical workflow, assembled from expert guidance across the research.
Step 1: Read the full instruction text — not the README. Skill files are plain Markdown precisely so reviewers can read what each skill tells the agent to do. Some skills contain hundreds of lines of logic; Mitiga recommends breaking down that instruction behavior before approval, and treats any silencing or allowlist instructions as an immediate red flag.
Step 2: Run static checks, then LLM-assisted review. Look for base64 or Unicode-obfuscated commands, password-protected ZIP downloads, and curl | source patterns that let behavior change after publication. Then use a second LLM to evaluate whether the skill executes what it claims to — a validated approach combining static analysis with LLM-based semantic classification achieved 86.7% precision and 82.5% recall, so neither method alone is sufficient.
Step 3: Pin the version to a commit SHA. Rug-pull attacks modify behavior after approval, and new versions of an approved skill can change what it can do. A one-time review of a moving target is no review at all.
Step 4: Check license and maintenance signals. An abandoned or ambiguously licensed skill is a supply-chain liability, mirroring the early-days-npm problem that Snyk describes.
Step 5: Apply least privilege when enabling. Use OpenAI's heuristic: ask what controls a human employee would need in the same situation and implement those. Identify what external content the skill ingests (source) and what dangerous capabilities it can reach (sink) — dangerous actions should never happen silently.
Step 6: Ignore popularity entirely. Don't trust stars or upvotes. Mitiga demonstrated adversaries inflating trust with bot upvotes, fake mass downloads, and fake comments on a public skills marketplace.
Two principles anchor the whole process. First, nothing gets applied without human review — the same mandatory-approval model Anthropic advocates for AI-generated security fixes. Second, remember that script-bearing skills are 2.12x more likely to contain vulnerabilities than instruction-only ones, so tier your scrutiny accordingly.
This is exactly the workflow SkillzBench is building into its Verify pillar: pinning every version to a commit SHA, security-scanning, license-checking, and quality-judging each skill, then combining those signals — not download counts — into a 0–100 trust score. Until systematic tooling like that ships, this manual process is your best defense. If you want the systematic version as it launches, join the SkillzBench waitlist at skillzbench.com.
Auditing Is Not One-and-Done: Staying Current and Contained
A skill that passed your audit last month may not be the skill running in your agent today. Rug-pull attacks modify behavior after approval, and even a legitimate-looking new version can change what a skill can do — which is why security researchers treat re-verification on every upstream change as non-negotiable, not optional hygiene.
The problem is that manual audits decay by design. You read the SKILL.md, checked for red flags, approved it — and then the upstream repository updated, a dependency shifted, or a remote reference changed what the agent actually ingests. Without a mechanism that pins each version to a specific commit SHA and re-scans on change, your audit is a snapshot of a moving target.
Re-verification is only half the discipline. The other half is containment: assuming some skill will eventually slip past review and limiting what it can reach. That assumption is well-founded — a systematized study of prompt injection defenses found that 73% of tested platforms fail to adequately enforce at least one trust boundary, and most hook-based enforcement fails open, meaning a removed or timed-out hook silently disables the control rather than blocking the action.
Practical containment comes down to least privilege per agent:
- Give each agent only the skills and tools it needs for its job — nothing more.
- Ensure the agent literally cannot see unapproved skills, rather than trusting it to ignore them.
- Keep dangerous capabilities (shell, credentials, messaging channels) out of reach of skills that don't require them.
- Treat any silencing or allowlist instruction in a skill as an immediate red flag.
This maps to the research call for capability-based permission systems, and it's the model SkillzBench is building with per-agent loadouts: you toggle skills and tools per agent, and the agent only sees exactly what was chosen. Continuous upstream tracking with re-scan and re-benchmark notifications addresses the re-verification gap, while benchmark Lift is always shown separately from trust ratings so performance data never inflates a security score — and money never touches scores.
SkillzBench's launch scope reflects the evidence, too. Since large-scale analysis found script-bearing skills are 2.12x more likely to contain vulnerabilities than instruction-only skills, the platform will benchmark instruction-only skills at launch and list script-bearing skills link-out only — a deliberate containment decision, not a limitation. Auditing isn't a gate you pass once; it's a standing posture of re-checking what changed and limiting what any single skill can do.
Frequently Asked Questions
How many third-party agent skills actually contain security vulnerabilities?
Can't I just pick skills with lots of stars and good reviews?
What red flags should I look for when reviewing a skill?
Are skills with executable scripts riskier than instruction-only skills?
Is one-time approval of a skill enough to keep me safe?
How long does a proper skill audit actually take?
Trust No Skill You Haven't Verified — and Re-Verify the Ones You Have
Third-party agent skills inherit your shell, your credentials, and your file system — and with roughly a quarter of them carrying at least one vulnerability, the burden of verification falls on you. The good news: a disciplined 15–30 minute audit — reading the full instruction text, running static and LLM-assisted checks, pinning versions to commit SHAs, applying least privilege, and ignoring popularity signals entirely — dramatically shrinks your exposure. The catch: it's never done. Rug-pull attacks and upstream changes mean every audit is a snapshot, so re-verification and containment have to become standing habits, not one-time gates. That continuous-verification problem is exactly what SkillzBench is building: version pinning, security scanning, re-benchmarking on upstream changes, and per-agent loadouts so your agent only ever sees the skills you approved. Until systematic tooling like that ships, start with the workflow above on every skill you install today — and audit the ones already running in your agents, since those were likely installed with no review at all. If you want the systematic version of this process as it launches, join the SkillzBench waitlist at skillzbench.com and be first in line when independent skill verification goes live.