Question 1

What does a priority benchmark run cost and how does pricing work?

Accepted Answer

SkillzBench is pre-launch, so exact pricing is still being finalized. The current plan includes a generous free tier, Pro at roughly $9-15/mo, Teams at roughly $15-25/seat/mo, and Creator at roughly $5-10/mo, with benchmark-as-a-service sold as priority runs. Join the waitlist to get pricing the moment it's announced.

Question 2

Can SkillzBench benchmark skills that include scripts, or only instruction-only SKILL.md packages?

Accepted Answer

At launch, SkillzBench benchmarks instruction-only skills: SKILL.md packages plus text references. Script-bearing skills are listed link-out only, meaning they appear in discovery but don't go through the sandbox Lift measurement pipeline.

Question 3

How does SkillzBench keep benchmark results independent if authors pay for priority runs?

Accepted Answer

SkillzBench enforces a strict separation: paying for priority runs affects only how fast your benchmark executes, never the outcome. Lift is always displayed separately from ratings, and money never touches scores, so a paid run cannot buy a better result.

Question 4

What happens when a skill I've benchmarked gets updated upstream?

Accepted Answer

SkillzBench's Stay Current pillar continuously tracks upstream changes, re-scans and re-benchmarks skills when they change, and sends notifications. Since every version is pinned to a commit SHA, you can see exactly which version each Lift result belongs to and compare performance across updates.

Question 5

How is this different from just reading a skill's GitHub stars or community reviews before adopting it?

Accepted Answer

Community signals are only one input. SkillzBench adds empirical Lift measurement from sandbox testing with the same model, plus security scans, license checks and quality judgments — things stars can't tell you. Published research found roughly of skills carry at least one vulnerability, which popularity metrics completely hide.

Benchmark-as-a-Service for Students