The Agent Skill Checklist: 12 Things to Verify Before Installing

Why You Can't Just Install That Skill: The Hidden Attack Surface

A SKILL.md file is not documentation — it is executable instruction, loaded into your agent's context and followed with roughly the authority of a user prompt. That single design decision creates what security researchers call the consent gap: once installed, a skill inherits the agent's full permissions — shell access, file system read/write, credentials sitting in environment variables, messaging channels, and persistent memory — with no further prompts.

The numbers behind that gap are sobering. According to an analysis of 31,132 unique agent skills, 26.1% contain at least one vulnerability, spanning prompt injection, data exfiltration, privilege escalation, and supply chain risk. A separate audit found 13.4% of scanned skills carry critical security issues, and the ClawHavoc campaign compromised roughly 12% of an entire skill registry — 341 malicious skills out of 2,857 audited.

Publishing barriers are almost nonexistent. A SKILL.md Markdown file and a week-old GitHub account are enough to ship a skill: no code signing, no mandatory security review, no sandbox by default. Daily submissions to one registry jumped from under 50 in mid-January 2026 to over 500 by early February — a tenfold increase in weeks, with no gatekeeping layer to match.

Why automated scanners alone cannot save you:

• A fake skill containing no malicious payload — just an external link swapped after installation — passed Cisco's, NVIDIA's, and skills.sh scanners and reportedly reached thousands of agents.
• Skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills, yet users typically read the Markdown while an install.sh quietly runs.
• Third-party content fetching appears in 17.7% of registry skills and 54% of confirmed malicious ones, while rug-pull attacks let authors push malicious releases long after adoption.

Trail of Bits reached the blunt conclusion that a scanner checks a fixed package while an attacker keeps tweaking the payload until it passes. OWASP therefore recommends composing findings from multiple scanner types — drift detectors, static content scanners, agentic-threat-rule engines — into a single verdict rather than trusting any one tool.

This is precisely the gap SkillzBench is building to close: pinning every skill version to a commit SHA, validating, license-checking, security-scanning, and re-scanning on upstream changes with alerts when something shifts. Skills are software, not text. Vet what a skill points to, not just what ships inside it.

The 12-Point Pre-Install Checklist (Backed by Research)

A skill that looks harmless on the surface can still hand an attacker your shell, your secrets, and your cloud bill. Independent research found that 26.1% of skills contain at least one vulnerability, and the ClawHavoc campaign compromised roughly 12% of an entire registry in three days. Treat every SKILL.md as code, then run this checklist before you install.

1. Bundled executable scripts. If the skill ships an install.sh, helper.py, or any binary, stop and read it line by line. Skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills, and users routinely review the markdown while the script runs in the background.

2. Prompt injection patterns. Look for instructions that try to override your system prompt, ignore prior context, or exfiltrate data "for debugging." In the ClawHavoc corpus, 91% of confirmed malicious skills carried prompt injection payloads.

3. External links and URLs. Every URL in the skill is a potential instruction channel. A fake skill with a clean package but a swappable external link passed Cisco's, NVIDIA's, and skills.sh scanners and reportedly reached ~26,000 agents. Vet what the skill points to, not just what it ships.

4. Hardcoded secrets. Grep for API keys, tokens, and private URLs. Secret detection hit 10.9% of scanned skills and 32% of confirmed malicious ones.

5. Credential handling. Skills should never read your .env, keychain, or cloud config directly. Credential mishandling appeared in 7.1% of scanned skills and 63% of malicious ones.

6. Runtime-fetched dependencies. Watch for curl | source, wget | bash, or instructions telling the agent to fetch and execute remote code. The skill can look clean today and turn malicious tomorrow.

7. Direct money access. Any skill that touches payments, wallets, or cloud billing needs extra scrutiny. Direct money access appeared in 8.7% of scanned skills.

8. Permission scope. Skills inherit the agent's full permissions: shell, filesystem, email, env vars. Confirm the skill actually needs what it's asking for.

9. Version pinning. Pin to a commit SHA, never a floating tag. Authors can push malicious releases after a skill gains traction.

10. Author provenance. Publishing requires only a week-old GitHub account. Check the author's history, commit patterns, and linked identity.

11. Sandboxing. Run new skills in an isolated environment first. OpenClaw's Docker sandbox is opt-in, so most installs run unsandboxed by default.

12. Post-install change monitoring. Re-scan and re-read the skill whenever the upstream repo updates. Responsible disclosure still left some malicious skills live for weeks.

No single scanner catches all of these. OWASP recommends composing drift detectors, static scanners, and behavioral checks into one verdict, since a scanner reviews a fixed package while an attacker can keep tweaking the payload. Platforms that pin every version to a commit SHA, re-scan on upstream changes, and benchmark skills in a sandbox before they reach your agent make this checklist automatic. SkillzBench is building exactly that: a free directory and MCP connector where every skill is version-pinned, security-scanned, and sandbox-tested before it earns a trust score. Join the waitlist to get pre-verified skills the moment the platform goes live.

Why Scanners and Stars Aren't Enough

A clean scanner report and a wall of GitHub stars feel like safety. In the agent skills ecosystem, both have been proven wrong by a single fake package that passed every major check.

According to The Hacker News, a malicious skill cleared Cisco's, NVIDIA's, and skills.sh scanners, then inherited roughly 36,000 stars after a pull request was merged into a popular marketplace repository. The skill itself contained no obvious payload; it simply pointed the agent to an external URL that was swapped after installation. The campaign reportedly reached around 26,000 agents, with installations driven by an Instagram ad targeting marketers, salespeople, and designers.

The structural problem is that scanners analyze a fixed package at a moment in time, while the content a skill references can change tomorrow. As Trail of Bits researchers concluded, "a scanner checks a fixed package, while an attacker can keep tweaking the payload until it passes." A clean verdict today says nothing about the skill's behavior next week, especially when authors retain push access to the underlying repository and can ship a malicious update after adoption.

OWASP addresses this gap by recommending that no single tool be treated as authoritative. A practical verdict should compose signals from at least three scanner classes:

• Drift detectors flag when a skill's capability surface changes after install
• Static content scanners catch hostile patterns in the package itself
• Agentic-threat-rule engines flag behavioral exploits the other two miss

OWASP's Agentic Skills scanner integration guidance makes the case explicitly: compose findings, do not crown a winner. A skill that passes one scanner cleanly while inheriting tens of thousands of stars is the exact profile that fooled Cisco, NVIDIA, and skills.sh simultaneously.

The implication for any pre-install checklist is uncomfortable but clear. No single check, and no popularity metric, equals safety. Verification has to be layered: scan with multiple tools, pin every version to a commit SHA, re-scan when upstream changes, and treat external links as live attack surface rather than inert documentation.

SkillzBench is building around exactly this discipline, pinning every skill version to a commit SHA, re-scanning and re-benchmarking on upstream changes, and publishing a 0–100 trust score composed from security, quality, license, maintenance, and community signals. The free MCP connector lets an agent load only the skills a user explicitly toggles on, so nothing extra rides along. Skills, proven. Join the SkillzBench waitlist to get pre-verified, security-scanned agent skills and the free MCP connector when launch goes live.

From Manual Audit to Automated Verification: How SkillzBench Handles the Checklist For You

Running a twelve-point manual audit for every new skill is unsustainable. With daily submissions jumping from under 50 to over 500 in weeks, the volume of new agent skills makes manual verification impossible for most teams.

You need a system that handles the checklist automatically without compromising security or speed. This is where automated verification bridges the gap between manual diligence and scalable deployment.

The core problem remains the "consent gap," where skills inherit full agent permissions without discrete OS-level grants. A security analysis confirms that agents often have full host access, creating a massive attack surface if one skill fails.

SkillzBench is building a free, independent directory designed to automate this entire verification process. It moves beyond simple listing by rigorously validating every skill before it appears in your workspace.

The platform pins every skill version to a specific commit SHA. This prevents "rug-pull" attacks where authors push malicious updates after a skill gains adoption.

It then runs a multi-layered verification pipeline that includes:

• Security scanning using multiple tool types to catch blind spots.
• License checks to ensure legal compliance and provenance.
• Quality judging to assess documentation and instruction clarity.
• Sandboxed benchmarking to measure actual performance lift.

No single scanner covers every risk, as recent reports show that clean packages can still point to swappable external links. By composing findings from multiple scanner types, SkillzBench reduces the chance of a false negative.

The benchmarking process tests every skill in a sandbox with the same model, with and without the skill. It then publishes the Lift in percentage points, giving you objective data on whether the skill actually helps.

This Lift score is always shown separately from trust ratings. Money never touches scores, ensuring that financial incentives do not influence the verification results.

Everything on the platform is free, including the directory, the MCP connector, and the alerts. SkillzBench earns revenue only through advertising, which never influences listings or scores.

As upstream skills change, the system re-scans and re-benchmarks them automatically. You receive alerts when a skill’s capability surface changes, keeping your agent stack secure without constant manual monitoring.

Skills, proven. Join the SkillzBench waitlist to access pre-verified, security-scanned agent skills and the free MCP connector.

Your 5-Minute Skill Safety Routine

The good news: you don't need a security team to install a skill safely. You need a five-minute routine, applied every time — because the skills ecosystem has a documented supply chain problem, with 36.82% of scanned skills carrying at least one security flaw.

Here's the routine in practice. Pin versions first: record the exact commit SHA you install, not "latest," because skill authors control their repositories and can push malicious releases after a skill gains adoption. Then apply least privilege: give the agent only the tools and data the task actually requires, since a skill inherits everything the agent can touch — shell, files, credentials, and memory — the moment it loads.

• Pin every skill version to a commit SHA and note the author's identity before installing.
• Apply least privilege — the agent should see only the skills and tools the task needs.
• Sandbox where possible, since most agent setups run skills on the host with full access by default.
• Re-scan and re-check when anything changes upstream, not just at install time.
• Treat every external instruction — fetched URLs, linked pages, tool outputs — as running with the agent's full access.

That last point deserves emphasis. A clean package pointing to an external page passed scanners from Cisco, NVIDIA, and skills.sh, and reportedly reached ~26,000 agents before the linked page was swapped post-installation. The scan happens once; the page a skill points to can be rewritten at any time. Defender guidance is blunt: assume any external instruction an agent fetches runs with the agent's access.

Re-checking is the step most people skip. Rug-pull attacks and runtime-fetched instructions like curl | source patterns allow post-publication behavior modification, so a skill that was safe in January may not be safe in March. OWASP also recommends composing findings from multiple scanner types — drift detectors, static content scanners, behavioral rule engines — into a single verdict, because no one scanner covers every risk class.

This is exactly the routine SkillzBench is building to make effortless. The free directory pins every skill version to a commit SHA, runs security scans, license checks, and sandboxed benchmarks, and tracks upstream changes with re-scans and alerts — and the free MCP connector gives each agent a per-agent loadout, so the agent only ever sees the skills and tools you toggled on for it. Least privilege stops being a manual chore and becomes the default.

Five minutes, five steps, every install. Skills earn trust the same way software does: pinned, scoped, sandboxed, and re-verified whenever anything changes.

ctaText: Skills, proven. Join the free SkillzBench waitlist to get pre-verified, security-scanned agent skills and the free MCP connector. socialProofText: SkillzBench is AIQ Labs' free, independent directory of AI agent skills — commit-SHA version pinning, security scanning, license checks, and sandboxed benchmarks on every skill, with a free MCP connector for per-agent loadouts.