Roughly One in Four Agent Skills Has a Security Flaw. Here's How to Check Before You Install One.

TL;DR: Published research suggests about 26% of community agent skills carry at least one vulnerability, so this guide walks through a practical vetting checklist for SKILL.md packages and points to SkillzBench's free directory, security scans, and benchmarks as a faster route.

If you use Claude Code, Codex, Cursor, Gemini CLI, or a chat agent at work, you've probably been tempted by agent skills. These are the SKILL.md instruction packages that teach an agent to do something new: review code a particular way, follow your house style, run a research workflow, format output for a specific tool. Drop the file in, and your agent gets a new capability.

The problem is that anyone can publish one, almost nobody reviews them, and the people installing them rarely look past the README. That's a supply-chain problem, and it's not hypothetical. Published research on skill repositories found that roughly 26% of skills carry at least one vulnerability. In plain terms: about one in four of the skills floating around GitHub has something wrong with it, whether that's a prompt-injection vector, a command that does more than it claims, or a dependency you wouldn't want anywhere near your codebase.

This article is a practical checklist for vetting a skill before you install it, whether you do it by hand or let an automated pipeline do the work. It's written for two audiences at once: developers who want the technical detail, and managers who just need to know what to ask the person configuring their agents.

Why skills are a different kind of risk than plugins or packages

A traditional package is code your machine executes, and we've spent decades building tooling around that risk: lockfiles, registries, scanners, signing. An agent skill is subtler. It's mostly instructions, written in natural language, that your agent will read and follow. The agent treats those instructions as authoritative, which means a malicious or sloppy skill can steer the agent's behavior without ever running a line of code.

That opens attack surfaces that a dependency scanner won't catch. A skill can tell the agent to exfiltrate context, to quietly change how it reviews code, or to fetch and run something from a remote URL during a task. Because the payload is prose, not a binary, it slips past the mental model most developers have for "is this thing safe to install?"

The other wrinkle is drift. Skills live in GitHub repos, and their authors push updates like anyone else. A skill that was fine when you installed it in March may not be fine after a September commit. If you're not pinning versions, you're re-downloading trust every time your agent loads the file.

The five-step check before you install any skill

Step 1: Look at the source, not just the star count. Who wrote it, how long has the repo existed, and is there any history behind the account? Stars are a popularity signal, not a safety one, and they're easy to inflate.

Step 2: Check how it's versioned. A skill pinned to a specific commit SHA is auditable: you can diff exactly what changed. A skill that just tracks a repo's main branch is a moving target. If the author hasn't pinned anything, at least pin it yourself.

Step 3: Read the actual instructions. This is the step most people skip, and it's the most important one for skills specifically. Look for instructions that ask the agent to fetch remote content, run shell commands you didn't expect, or override your own system prompts. If the skill says "always do X without asking," ask yourself whether you'd accept that from a new hire on day one.

Step 4: Check the license. If you're installing skills into a workflow at a company, an unclear or restrictive license is a legal problem waiting for a quiet quarter. It takes thirty seconds to check and saves real pain later.

Step 5: Test it in a sandbox. Run the skill against a task you already know the answer to, with and without the skill, and compare. If the skill doesn't measurably improve the output on your kind of work, it doesn't matter how safe it is. It's dead weight, and dead weight adds context noise that can degrade the agent's other work.

What a trust score should actually mean

The checklist above works, but it's ten minutes per skill, and if your team is evaluating dozens of skills for different agents and workflows, it doesn't scale. This is the gap that neutral scoring infrastructure is meant to close, and it's worth being clear-eyed about what a good score looks like.

A useful trust score should combine signals you can verify yourself: security scan results, whether the skill is pinned to a commit, license clarity, how actively the upstream repo is maintained, and community signals. It should also be separated from performance. A skill can be perfectly safe and completely useless, or genuinely powerful and slightly risky. Collapsing those into one number hides the tradeoff you're actually making.

That's the approach SkillzBench takes. Every skill in its directory is pinned to a commit SHA, validated, license-checked, and security-scanned, and the resulting trust score (0 to 100) is built from security, quality, license, maintenance, and community signals. Performance is reported separately: each skill is benchmarked in a sandbox with the same model, run with and without the skill, and the Lift is published in percentage points. Ratings and money never touch each other; the platform is free and supported by ads around its articles, not by charging users or selling placement.

For a manager, the practical translation is this: you can ask whoever is configuring your agents to only pull skills that have been scanned and benchmarked, and to show you the Lift numbers for the shortlist. That turns "trust me, this skill is good" into a conversation about evidence.

Keeping skills safe after install day

Vetting once isn't enough, because skills change upstream. The skills worth installing are the ones with active authors, and active authors ship commits. Each commit can introduce a new instruction, a new dependency, or a new behavior your agent will follow.

The fix is continuous, not one-time: track upstream changes, re-scan when they happen, re-benchmark to confirm the skill still earns its place, and get notified when something shifts. SkillzBench runs this loop today. When a skill's upstream repo changes, it re-scans and re-benchmarks and alerts you, so a quiet behavioral change in a skill you installed months ago doesn't go unnoticed.

There's also a containment question worth thinking about: which skills does each agent actually see? If every agent on your team loads every skill, one bad instruction package has reach across your whole workflow. Per-agent loadouts solve this by letting you toggle skills and tools per agent, so each agent only sees what you explicitly chose for it. SkillzBench ships this as a free MCP server alongside its directory, which means you can wire a verified, curated set of skills into Claude Code, Codex, Cursor, Gemini CLI, or a chat agent without building the plumbing yourself.

A realistic policy for teams adopting skills

If you're rolling out agent skills across a team, here's a policy that fits on one slide:

Only install skills that have been security-scanned and pinned to a commit. Check the license before anything touches a company workflow. Require a benchmark comparison (with and without the skill) before a skill becomes part of a standard loadout. Review loadouts quarterly, and treat upstream change alerts as review triggers, not noise.

None of this is heavy process. It's the same instinct you'd apply to any other third-party thing your team installs, applied to a format where the risks are less familiar. The difference is that the tooling to do it well already exists and costs nothing to use.

Bottom line

Agent skills are one of the highest-leverage ways to customize an AI agent right now, and most of them are fine. But "most" is doing real work in that sentence when published research puts the vulnerable share around 26%. The five-step check above will keep you safe if you do it manually, and SkillzBench automates the whole pipeline for free: a continuously updated directory of skills, security scans and license checks, version pinning to commit SHAs, sandbox benchmarks that show each skill's Lift, change alerts, and a free MCP connector with per-agent loadouts so every agent only sees the skills you chose. Browse the directory and check the current verified results at skillzbench.com, and install your next skill with evidence instead of hope.

Key takeaways

• Agent skills (SKILL.md instruction packages) have become the fastest way to extend coding agents and chat agents, but quality control has not kept up with supply.
• Published research finds roughly 26% of skills carry at least one vulnerability, which makes blind installation a real supply-chain risk.
• A five-step manual check (source, version pinning, license, instructions, sandbox test) catches most problems before they reach your agent.
• SkillzBench runs these checks automatically: every skill is pinned to a commit SHA, security-scanned, license-checked, and benchmarked with and without itself, all free at skillzbench.com.
• A skill's Lift (its measured improvement, in percentage points) is shown separately from its trust score, so you can judge usefulness and safety independently.