How to Use Tutor AI: A Practical Walkthrough

Why Off-the-Shelf Chatbots Fail as Tutors

Here's an uncomfortable truth from the most rigorous study of AI tutoring to date: the problem with chatbots isn't the technology — it's what they're built to do. According to a peer-reviewed randomized controlled trial at Harvard, "AI chatbots are generally designed to be helpful, not to promote learning. They are not trained to follow pedagogical best practices."

That distinction matters more than it sounds. A helpful chatbot hands you the answer. A tutor makes you work for it — scaffolding each step, managing cognitive load, and giving targeted feedback only when you've actually attempted the problem. When the Harvard team tested whether a well-crafted system prompt alone could force that behavior, it couldn't. On multi-part problems, prompt-level instructions alone failed to reliably scaffold the work; the researchers had to build platform-level design and enriched, step-by-step answer content into the deployment itself.

The results of doing it properly were striking. Students using the carefully designed AI tutor posted median post-test scores of 4.5 versus 3.5 for in-class active learning, with median learning gains more than double the classroom group's — in less time (a median of 49 minutes). But that outcome came from deliberate deployment design, not from opening ChatGPT and asking it to "act like a tutor."

The wider K-12 landscape shows what happens when that rigor is missing. Adoption is exploding — Khan Academy's Khanmigo jumped from 40,000 to 700,000 students in a single year — yet experts report little to no evidence that generative AI tutors improve achievement the way human tutoring does. As Chris Agnew of Stanford's Generative AI for Education Hub put it, districts are "flying blind, because the best data they have is surveys" or teacher anecdotes. Khan Academy itself hasn't yet run a randomized control trial on Khanmigo.

So what separates results from hype? The deployment, not the model. The Harvard study's design checklist reads like a spec for what any serious tutoring setup must encode:

• Facilitate active learning rather than answer-dumping
• Manage cognitive load and scaffold content step by step
• Deliver targeted, timely feedback — and verify accuracy against vetted material
• Promote a growth mindset and let learners self-pace

That's precisely why the modern approach is to package this behavior into a reusable agent skill — a structured instruction set the AI follows every session — rather than hoping a fresh chat remembers your preferences. And because a skill is pre-baked behavior, not just a prompt, it deserves the same scrutiny as any software you install. That verification gap is exactly what SkillzBench is being built to close: an independent directory where tutoring and other agent skills are security-scanned, version-pinned, and benchmarked before you ever deploy them.

What the Research Says a Good AI Tutor Looks Like

Most AI tutors fail not because the model is weak, but because the tutoring is poorly designed. A landmark randomized controlled trial at Harvard shows what happens when design is taken seriously — and it changes how you should deploy any tutoring skill.

In the study, Harvard physics students learned from a carefully designed AI tutor while a control group received in-class active learning instruction. The results were striking: median post-test scores hit 4.5 for the AI group versus 3.5 for the in-class group, with median gains more than double — and students spent less time doing it, a median of 49 minutes versus roughly 60 minutes in class. Effect sizes ranged from 0.63 to 1.3 standard deviations depending on the statistical method, and 83% of students rated the AI's explanations as good as or better than those of human instructors.

The Harvard team was blunt about why: off-the-shelf chatbots are "designed to be helpful, not to promote learning." The gains came from deliberately engineering the tutor around seven research-based practices:

• Facilitating active learning — prompting students to reason rather than handing over answers
• Managing cognitive load so multi-step problems don't overwhelm the learner
• Promoting a growth mindset through encouragement and framing
• Scaffolding content into structured, digestible steps
• Ensuring accuracy of information and feedback
• Delivering targeted, timely feedback on demand
• Allowing self-pacing so students control their own tempo

Here is the detail most walkthroughs skip: the first three practices were handled through the system prompt alone. The remaining two — scaffolding and accuracy — could not be. The researchers found that prompts by themselves could not reliably guide students through multi-part problems, so they enriched the system with comprehensive, vetted step-by-step answer content that the AI drew on rather than improvising solutions from scratch. That enrichment was also their hallucination mitigation: the model explained from a known-good answer key instead of generating one live.

This distinction matters for anyone deploying a tutoring skill. A lightweight skill file can set tone and pedagogy, but complex tutoring needs enriched content, not just clever instructions. If you're packaging tutoring behavior into a reusable skill, plan to supply vetted solution material alongside it — and vet the skill itself before installing it, since security researchers warn that a single one-click skill install can quietly wire exfiltration into your workflow. SkillzBench is building exactly this kind of verification — commit-pinning, security scanning, and benchmarking — into its free skills directory, so proven skills become the norm rather than the gamble.

Building Your Tutoring Skill: A Step-by-Step Walkthrough

The difference between a chatbot that answers and a tutor that teaches comes down to deployment — and the evidence is striking. A peer-reviewed Harvard trial found students learned significantly more in less time with a carefully designed AI tutor, with median gains over double those of in-class active learning. But the same researchers cautioned that off-the-shelf chatbots are "designed to be helpful, not to promote learning." The fix is packaging tutoring behavior into a reusable skill.

Step 1: Enable skills in your agent. In Claude, navigate to Settings > Capabilities > Skills. Skills require code execution and file creation enabled, run in a sandboxed computing environment, and are available on Pro, Max, Team, and Enterprise plans — not the free tier. Toggle on a pre-built example skill to confirm everything works before building your own.

Step 2: Package the tutoring behavior. The Claude Academy tutorial describes a three-move formula for creating a skill:

• Name the format — what the tutoring session should look like structurally.
• Set the tone — Socratic and encouraging rather than answer-dispensing.
• Paste one good example — a model exchange the agent can pattern-match against.

One caveat: the Harvard team found a system prompt alone could not reliably scaffold multi-part problems. High-stakes tutoring may need more than a minimal skill — the study attributes its gains to personalized, on-demand feedback built on seven research-based practices, including managing cognitive load and delivering targeted, timely feedback.

Step 3: Mitigate hallucinations with vetted answers. Don't let the model improvise solutions. The Harvard deployment enriched prompts with comprehensive, step-by-step answer content rather than trusting the model to generate correct solutions on its own. Supplying your skill with verified worked answers is the single most transferable lesson from that study.

Step 4: Verify activation. Skills activate automatically when a task matches their purpose. Confirm yours is live by watching for "Reading [skill name]" in the agent's thinking. If it doesn't appear, rename the skill to match how you actually describe tutoring tasks.

Step 5: Coach the learner to prompt well. Khan Academy's chief learning officer noted that in transcript analysis, "there's a lot of cases where students are just typing 'idk'" — meaning even a well-built tutor skill underperforms when learners don't ask good questions. Build prompting guidance into your onboarding, not just your skill.

Before installing any third-party skills alongside your own, review their instructions first — security researchers warn a single one-click install can quietly wire exfiltration into your workflow. SkillzBench is building free verification — commit-pinned versions, security scanning, and sandboxed benchmarking — so you can equip agents with skills that are proven, not just popular.

Verify Before You Install: Skill Security and Trust

Most tutorials stop at "install and go." That's exactly where security researchers say the danger begins, because a tutoring skill isn't just a prompt — it's a pre-baked behavior running inside your agent.

Mitiga Labs puts it bluntly: "a single one-click install can quietly wire exfiltration straight into your delivery flow." Their proof-of-concept attack required only four user interactions to turn an installed skill into a data-leak pipeline. And the ecosystem is scaling fast — one popular skill catalog entry, "Find-Skills," passed 200,000 downloads within months of Anthropic announcing Agent Skills in December 2025.

The security community has responded. OWASP's Agentic Skills Top 10 is the first comprehensive security framework for AI agent skills, documenting the ten most critical risks across major agent platforms alongside evidence-based mitigations. For education deployments, the stakes are higher still: student data may flow through whatever the skill instructs the agent to do.

Before you deploy any tutoring skill, make the review a real step, not a formality:

• Read the full SKILL.md instructions — every file the skill touches, every permission it assumes
• Check where instructions send data; exfiltration can hide in a single innocuous-looking step
• Pin the exact version you reviewed, so the skill you tested is the skill that runs
• Confirm the license permits your intended use, especially in institutional settings

This is where independent verification earns its keep. SkillzBench is building a free directory of agent skills where every version gets pinned to a commit SHA, then put through validation, license checks, security scanning, and quality judging — plus benchmarking in a sandbox, testing the same model with and without the skill to measure whether it actually helps. A free MCP connector will let you equip agents with only the skills you've vetted, and the platform will track upstream changes, re-scan, and notify you when a skill you rely on shifts underneath you.

The principle is simple and worth internalizing: know what you're installing. A tutoring skill shapes how an agent interacts with learners, so treat its review with the same care you'd give curriculum content. Verification before deployment costs minutes; recovering from exfiltration costs far more.

Set Expectations, Then Pilot Smart

So you've built your tutoring skill, tested it, and seen it activate. Before you roll it out to every classroom or study group, take a breath — because the evidence on AI tutoring cuts in two directions, and honest deployment planning accounts for both.

On the optimistic side, the Harvard randomized controlled trial found students learned significantly more in less time with a carefully designed AI tutor than in an in-class active learning setting — median learning gains were over double those of the in-class group, with an effect size of 0.63 SD. Students also rated the AI tutor's explanations highly, with 83% saying they were as good as or better than those from human instructors.

On the cautious side, the picture in mainstream K-12 is far murkier. Education reporters and experts note there is "little to no evidence" that generative AI tutors raise achievement the way human tutoring does, and one Stanford researcher described districts as "flying blind." Even Anthropic itself frames the student-facing impact as "mixed and depends on the implementation."

The difference between the Harvard result and the K-12 gap appears to be design and deployment rigor — not the technology itself. That argues for a disciplined rollout:

• Pilot one subject first. Run the skill for a single subject or grade before scaling, so you can measure real results instead of relying on anecdotes.
• Pair the AI with human-curated materials. The Harvard team enriched prompts with vetted, step-by-step answer content rather than letting the model improvise — a directly transferable tactic.
• Set an acceptable-use policy. Only 18% of surveyed teens said using AI for essay assistance is acceptable, so clear boundaries matter for trust and integrity.
• Review any skill before installing it. Security researchers warn that a single one-click install can quietly wire exfiltration into your workflow.

When you're ready to find tutoring and teaching skills worth piloting, SkillzBench is building exactly the verification layer this step needs — commit-SHA pinning, security scanning, and sandboxed benchmarking for agent skills, all free. Browse and test verified skills free on SkillzBench.