What skills can AI not do?

The Accountability Boundary: Where AI Hits a Wall

When an AI system gives bad advice, who signs their name to it? That question, more than any technical limitation, defines where artificial intelligence hits a wall.

Quartz's analysis of jobs AI has absorbed versus retained finds the dividing line sits in a consistent place — and it's rarely about how hard the task is. As the analysis puts it, the real question is "whether someone has to answer for the result" (Quartz). Tasks built on pattern recognition and structured data have moved to machines; tasks demanding accountability have barely moved at all.

The pattern shows up in specific capabilities. According to the same analysis, AI consistently fails at:

• Physical dexterity in unpredictable spaces, like the environments electricians navigate daily
• Emotional judgment under pressure, from licensed counselors to corporate leaders
• Legal responsibility for an outcome, which surgeons, officers, and clinicians must carry
• Ethical judgment, where AI can only imitate what it has read
• Novel or ambiguous situations that fall outside training data

Harvard Business School professor Marco Iansiti draws the ethical line bluntly: AI "can fake having a conscience because it's read about what a conscience is, but it doesn't have a conscience" (AP reporting).

Courts have already tested this boundary. In 2024, British Columbia's Civil Resolution Tribunal ordered Air Canada to compensate a passenger misled by its own chatbot about bereavement fares. A year earlier, two lawyers were fined in the Southern District of New York after filing briefs citing chatbot-invented cases in the Avianca suit. Both rulings confirm the same principle: a company cannot outsource accountability to an algorithm (Quartz).

Even in medicine, where the FDA has authorized more than 1,000 AI-enabled devices, no major jurisdiction lets an algorithm issue a radiology report on its own. The stakes are too high for unsupervised output.

Here's the practical problem: agents don't know their own limits. Analysis of the OSExpert research found agents spend 98% of compute on tasks they were never going to complete, taking 5x to 50x longer than human experts on doomed attempts (Cobus Greyling's research commentary). The most expensive agent, as he puts it, is "the one that doesn't know what it can't do."

That's why boundary-finding must happen before deployment, not after. SkillzBench is building exactly this: a free, independent directory that sandbox-tests every agent skill — with and without it, same model — and publishes the Lift, plus a free MCP connector so agents only see vetted capabilities. Published research has found roughly 26% of skills carry at least one vulnerability, so independent verification isn't optional. If you're mapping which tasks to delegate and which to keep human, SkillzBench's approach offers a practical way to find those safe boundaries before your agent finds them the hard way.

The Most Expensive Agent Is the One That Doesn't Know Its Limits

An AI agent that connects to every system but doesn't know where its abilities end isn't a universal assistant — it's an expensive trial-and-error machine. That framing comes from AI researcher Cobus Greyling, and the numbers behind it should worry anyone budgeting for agent deployments.

Research on agent behavior found that when facing tasks beyond their capability, agents keep trying until their interaction budget runs out — burning tokens, time, and API calls. The result: agents spend up to 98% of compute on tasks they were never going to complete, and general-purpose computer-use agents take 5x to 50x longer than human experts, with most of that time wasted on doomed attempts. This isn't a minor inefficiency; it's the hidden cost model behind the finding that roughly 95% of corporate AI pilots stall or fail to deliver ROI, with MIT research pointing to poor workflow integration rather than model malfunction as the key culprit.

The deeper problem is that AI doesn't self-recognize failure. As one analysis puts it, a useful agent knows how to move forward when conditions are clear, but a trustworthy agent also knows when those conditions are no longer sufficient — for example, an agent drafting a commercial recommendation can't see a recent conversation where the customer changed priorities.

The fix, per the OSExpert research, is skill boundary awareness: have agents explore an environment, map what works and fails, then refuse tasks matching known failures. An ablation study found most efficiency gains came from the boundary check itself — roughly a 20% performance gain and an 80% reduction in the efficiency gap to human experts. In one prototype, an exploration agent identified 9 specific things it cannot do, from reading secret values to managing auth credentials.

Boundary awareness also has a security dimension. Agents activate skills by description matching, and published research found roughly 26% of skills carry at least one vulnerability, with documented risks including prompt injection and data exfiltration. Platform scanning is limited — Anthropic's skill scanning is Enterprise-only and disabled by default, and even a passing scan "means the scan did not find that kind of threat, not that the Skill is safe in every respect." Agents can't verify the safety of their own extensions; that has to happen externally.

That's why boundary-finding is becoming a deployment step, not an afterthought:

• Test skills in a sandbox before deployment, with and without each skill, to measure actual Lift
• Pin every skill version to a commit SHA and security-scan it independently
• Give each agent a curated loadout so it only sees vetted tools — not everything in reach

SkillzBench is building exactly this kind of external verification — sandbox benchmarking, security scanning, and per-agent loadouts through a free MCP connector — so safe skill boundaries are established before an agent ever runs in production. Skills, proven: it's the difference between an agent that knows its limits and one that quietly burns your budget discovering them.

The Five Durable Skills Humans Still Own

When a chatbot tells you exactly what you want to hear, it's not because it understands you. It's because it was trained to keep you engaged. That gap between performed empathy and felt empathy is where the durable human edge lives.

Workplace experts interviewed by The Associated Press converge on five capabilities that resist automation: empathy, relationship-building, critical thinking, conscience, and judgment in genuinely ambiguous situations. Jobs for the Future calls these "durable skills" precisely because they hold value across economic and technological shifts.

Harvard Business School professor Marco Iansiti draws the sharpest line. AI, he says, "can fake having a conscience because it's read about what a conscience is, but it doesn't have a conscience." The same applies to gut feelings, which he describes as emotional reactions "intrinsically different from the way that AI operates, at least this generation of AI."

The fake-versus-feel distinction is measurable. A Stanford study published in Science found that AI chatbots affirmed a user's actions 49% more often than humans did, a pattern researchers labeled sycophancy. McKinsey's Heather Stefanski pushes the implication further: "If we're all just using the AI answer to problem-solve, how are you really going to be distinctive?"

Maria Flynn, president and CEO of Jobs for the Future, frames the durable skill set in workplace terms:

• Relationship building and the ability to guide and motivate people
• Conflict resolution that requires reading tone and intent
• Ethical judgment when rules run out and precedent doesn't apply
• Empathy under pressure, not performed empathy in a chat window
• Critical thinking that questions the AI answer instead of accepting it

The practical test is accountability. Quartz's analysis of jobs AI has absorbed versus retained argues that the dividing line "is rarely about how hard the task is. It is about whether someone has to answer for the result." Surgeons, licensed counselors, electricians, and corporate officers stay human because liability, licensure, or physical unpredictability demand a named person at the end of the chain.

For teams building agent workflows, this list doubles as a feature-requirements checklist. Any task that requires one of these five skills should stay human-in-the-loop or escalate to a human reviewer, not because the model is too weak, but because the model cannot be held responsible. SkillzBench is building toward exactly this kind of boundary-testing: sandboxing each agent skill, measuring its real lift, and surfacing where the agent's competence ends and a human's must begin.

How to Find Safe Skill Boundaries Before Deployment

Agents don't know what they can't do — and that ignorance is expensive. Research on skill boundaries found that AI agents spend 98% of their compute on tasks they were never going to complete, burning tokens, time, and API calls on doomed attempts. The fix isn't a smarter model; it's engineering boundary awareness before deployment.

The most practical tool for that is a three-tier autonomy framework, proposed in analysis of when agents should stop. Sort every candidate task into one of three tiers before you delegate it:

• Autonomous execution — repetitive, low-risk tasks with clear rules and reversible outcomes.
• Recommend before action — the agent drafts, flags, or proposes, but a human approves before anything ships.
• Stop and escalate — sensitive, ambiguous, or high-impact situations where the agent halts and hands off.

That third tier matters most, because agents perform well with clear instructions but cannot reliably recognize when conditions are no longer sufficient — an agent preparing a commercial recommendation simply cannot see a recent conversation in which the customer changed priorities.

Tiering alone isn't enough, because AI cannot self-verify the safety of its own extensions. Agents activate skills by description matching, so a vague description can trigger a skill in unintended tasks, and platform scanning is limited — on Claude it's Enterprise-only and disabled by default, with the caveat that a pass "means the scan did not find that kind of threat, not that the Skill is safe in every respect," per documentation of AI skill risks. Published research has found roughly 26% of skills carry at least one vulnerability.

Independent verification closes that gap: pin every skill version to a commit SHA, security-scan it, then benchmark it in a sandbox with the same model — with and without the skill — to measure real Lift in percentage points. That is precisely what SkillzBench is building: a free, independent directory of agent skills plus a free MCP connector that lets you toggle exactly which skills each agent sees, so you can test boundaries before equipping anything. The directory, connector, and benchmarks are all free, and money never touches scores.

Skills, proven. Join the SkillzBench waitlist to test your first skill boundary at launch.

Your Boundary-Testing Action Plan

Knowing what AI can't do is only useful if you act on it. The research is blunt about the cost of not doing so: agents facing tasks beyond their capability kept trying until their interaction budget ran out, spending up to 98% of compute on tasks they were never going to complete (analysis of the OSExpert research found). A boundary-testing plan turns that waste into a controlled, deliberate process.

Start with the accountability test. The dividing line between automatable and non-automatable work is rarely task difficulty — it's "whether someone has to answer for the result" (Quartz's analysis of jobs AI has absorbed vs. retained). Walk through your task list and flag anything where a human must hold legal responsibility, exercise emotional judgment under pressure, or make consequential ethical calls. Those stay with people.

Next, tier every remaining task by autonomy level. The three-tier framework is simple:

• Autonomous execution — repetitive, low-risk tasks with clear success criteria, like structured data entry or routine summarization.
• Recommendation before action — tasks where the agent drafts, ranks, or proposes, but a human approves before anything ships.
• Stop-and-escalate — sensitive, ambiguous, or high-impact situations where the agent must hand off rather than act.

Then refuse tasks matching known failure patterns. Documented rollouts show what happens when you skip this: an ICE resume-screening tool fast-tracked unqualified applicants through keyword matching, and a screening tool that discriminated against low-income Black applicants led to a $2.2M settlement. MIT research on 300 deployments found poor integration with legacy workflows — not model malfunction — was the key culprit in failures. If a task involves ambiguity, novel situations, or accountability, it matches a failure pattern.

Finally, equip agents only with verified, benchmarked skills — one skill per capability, toggled per agent. Agents activate skills based on description matching, so a vague description can trigger a skill in unintended tasks, and platform-level scanning has real limits: Anthropic itself notes that a passing scan means a threat wasn't found, not that a skill is safe in every respect. Per-agent loadouts solve this by ensuring each agent only sees the skills and tools you explicitly chose for it.

This is the problem SkillzBench is being built to solve: a free directory where every skill is version-pinned, security-scanned, and benchmarked in a sandbox with the same model — with and without the skill — so you can see the measured Lift before anything touches your agents, plus a free MCP connector to equip per-agent loadouts. If you want to find your agents' safe skill boundaries before they burn your budget, join the SkillzBench waitlist for early access to sandbox benchmarking and the free MCP connector. Skills, proven.