Question 1

What exactly does SkillzBench benchmark about an agent skill?

Accepted Answer

Every skill is tested in a sandbox with the same model, run both with and without the skill, and the difference is published as the Lift in percentage points. Lift is always shown separately from ratings, and money never touches scores, so the number reflects controlled measurement rather than promotion.

Question 2

How does SkillzBench handle the security risks in SKILL.md packages?

Accepted Answer

Every skill version is pinned to a commit SHA, validated, license-checked, security-scanned, and quality-judged before it earns a trust score from 0-100 built on security, quality, license, maintenance, and community signals. At launch, script-bearing skills are listed link-out only — only instruction-only skills (SKILL.md plus text references) are fully verified.

Question 3

Why can't I just rely on existing AI agent benchmarks like SWE-bench or WebArena?

Accepted Answer

Severe issues were found in 8 of 10 popular AI agent benchmarks, causing up to 100% misestimation of capabilities, and OpenAI dropped SWE-bench Verified after an internal audit found 59.4% of audited problems had flawed tests. SkillzBench takes a different approach: instead of relying on saturated leaderboards, it measures each skill's Lift directly in a controlled sandbox.

Question 4

When can I actually start using SkillzBench?

Accepted Answer

SkillzBench is pre-launch and actively being built — the waitlist and newsletter are live today, but the product itself is not yet public. Join the waitlist to be notified at launch and to get access to the Discover, Verify, Benchmark, and Equip pillars as they roll out.

Question 5

How is SkillzBench different from just browsing GitHub for agent skills?

Accepted Answer

GitHub gives you raw skills with no verification, no benchmarking, and no change tracking. SkillzBench continuously crawls and dedupes skills from GitHub, then pins versions, scans for vulnerabilities, quality-judges them, benchmarks their Lift, and re-scans when upstream changes — all through one MCP server with per-agent loadouts you control.

Agent Skill Benchmarking for Job Seekers