The Model Isn't the Bottleneck — Your Context Is

When an AI agent fails in production, the postmortem almost never blames the model. The real culprit is usually what the model was given to work with — the context. As Atlan puts it, "the model is not the bottleneck. The context it receives is."

Andrej Karpathy popularized the mental model that makes this concrete: the LLM is like a CPU, and its context window is like RAM — a finite working memory for the model. Context engineering, in his words, is the "delicate art and science of filling the context window with just the right information for the next step" (LangChain). Shopify CEO Tobi Lütke echoed the framing, preferring the term because it "describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM" (Addy Osmani).

The distinction matters because prompting and context engineering solve different problems. Prompt engineering is about how you phrase a question; context engineering is about constructing the entire information environment so the AI can solve the problem reliably. Or as Osmani puts it: if prompting was about finding a magical sentence, context engineering is about writing the full screenplay.

The numbers back this up. A 2025 study found that LLM accuracy drops 24.2% when relevant information sits buried in longer contexts — even when irrelevant tokens are masked. The model didn't get dumber; the information environment got noisier.

It gets worse as systems chain together. CMU research found that multi-agent systems relying solely on prompting fail roughly 70% of the time on multistep tasks. And reliability compounds unforgivingly: a chain of ten agents each performing at 95% yields only about 60% system reliability (0.95^10). Ten near-perfect steps still produce a failing system four times out of ten.

The failure modes have names (via LangChain):

• Context Poisoning — hallucinated content enters the context and contaminates downstream steps
• Context Distraction — an overloaded window overwhelms what the model learned in training
• Context Confusion — superfluous information skews the response
• Context Clash — contradictory pieces of context disagree with each other

This is why curation beats accumulation. Elastic's tool-orchestration principle is blunt: if a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better (Elastic). Agents need to see less, not more — which is the same principle behind SkillzBench's per-agent loadouts, where an agent only receives the skills and tools explicitly chosen for it.

The takeaway is uncomfortable but liberating: you can't prompt your way out of a context problem. The fix is engineering the information environment itself.

Where Prompting Breaks: Context Rot and the Failure Modes

Your prompt works fine — until it doesn't. The moment a task spans dozens of steps, tool calls, and hundreds of turns, the same model that summarized your document flawlessly starts forgetting, misfiring, and hallucinating.

Prompting still holds up well for self-contained tasks: summarization, extraction, translation, one-shot classification. The input is bounded, the goal is clear, and one well-crafted instruction usually suffices. But as Neo4j's analysis puts it, prompting breaks down in multi-step agents, long workflows, tool use, and dynamic task states — and the numbers are brutal. CMU research found multi-agent systems relying solely on prompting fail roughly 70% of the time on multistep tasks, and a 2025 study reported that LLM accuracy drops 24.2% when relevant information is buried in longer contexts.

The underlying phenomenon is context rot: overloading a model with long, unstructured text dilutes its attention. As the context grows, the model struggles to find the relevant parts — and every step of an agent's workflow adds more context, compounding the problem. Drew Breunig's framework, popularized by LangChain, names four failure modes:

• Context Poisoning — a hallucination enters the context and gets treated as fact from then on.
• Context Distraction — the accumulated context overwhelms what the model learned in training.
• Context Confusion — superfluous context influences the response in unhelpful ways.
• Context Clash — different parts of the context actively disagree with each other.

Elastic frames the same problem from a resource angle: the context window is finite, and mismanaging it fails in three ways — too little information produces hallucination, too much produces overflow, and conflicting information produces confusion. As Elastic's Tomás Murua puts it, "Prompt engineering takes the context window as given. Context engineering actively curates it."

Now, the honest objection: isn't this just rebranded prompting? Addy Osmani acknowledges that many experienced developers see context engineering as buzzword creation — and, as he admits, "these concerns aren't unfounded." Prompting Guide AI goes further, arguing prompt engineering is "now even more important than ever" and that context engineering should perhaps have been the name all along.

Both sides are actually right, because they're describing a subset relationship, not a rivalry. Prompt engineering is the craft of phrasing instructions well; context engineering is the discipline of constructing the entire information environment — which includes instructions, but also memory, tool selection, retrieval, and compression. A perfectly written prompt dropped into a poisoned, cluttered, contradictory context still fails.

This is also why curation beats accumulation in practice. Elastic's tool-orchestration principle is blunt: if a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better. It's a principle SkillzBench is building its whole Equip pillar around — per-agent loadouts where the agent only sees the skills and tools deliberately chosen for it, rather than everything bolted on at once. Curating the minimal viable context isn't a nice-to-have; it's the difference between an agent that compounds errors and one that compounds success.

The Engineer's Playbook: Write, Select, Compress, Isolate

So how do engineers actually do context engineering in practice? LangChain's canonical framework — Write, Select, Compress, Isolate — has become the industry's shared playbook, and it maps neatly onto what production teams are already shipping (https://www.langchain.com/blog/context-engineering-for-agents).

Write means giving the agent a place to think: scratchpads for intermediate reasoning and long-term memories that persist across sessions. Techniques like Reflexion and Generative Agents pioneered this, and products like ChatGPT, Cursor, and Windsurf now auto-generate memories for users.

Select is about curating what enters the window. The standout result here: applying RAG over tool descriptions improved tool selection accuracy by 3-fold in LangChain's work (https://www.langchain.com/blog/context-engineering-for-agents). The same logic applies to memory and knowledge retrieval — don't dump everything in, retrieve what's relevant to the current step.

Compress tackles the reality that agents run long. Claude Code triggers an "auto-compact" summarization pass after the conversation exceeds 95% of the context window. Cognition takes a different angle, using a fine-tuned summarization model specifically at agent-to-agent boundaries, so handoffs stay lean.

Isolate gives each sub-task its own clean window. OpenAI Swarm and Anthropic's multi-agent researcher exemplify this: sub-agents work in separate contexts, and Anthropic's researcher truncates anything exceeding 200,000 tokens. Multi-agent systems can consume up to 15× more tokens than chat, which is why isolation matters (https://www.langchain.com/blog/context-engineering-for-agents).

Underneath all four moves sits a broader shift: from pre-retrieval to just-in-time context. Traditional RAG pre-loads everything potentially relevant; newer patterns keep the window lean:

• Lightweight references — file paths, queries, document IDs — held in context instead of full content
• Dynamic loading at runtime, pulling data only when a task actually needs it
• Hybrid approaches that combine just-in-time loading with selective pre-retrieval

Anthropic's Agent Skills are the clearest example of this shift: a SKILL.md file acts as a lightweight pointer loaded only when relevant, rather than permanently occupying the window (https://www.elastic.co/search-labs/blog/context-engineering-vs-prompt-engineering). Elastic frames the guiding principle bluntly — curate the minimal viable set of tools, because "if a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better" (https://www.elastic.co/search-labs/blog/context-engineering-vs-prompt-engineering).

That curation problem is exactly why SkillzBench is building per-agent loadouts — so an agent only sees the skills and tools explicitly chosen for it, rather than everything installed. The same reasoning drives its benchmarking work: testing every skill in a sandbox, with and without it, to measure whether a SKILL.md package actually lifts performance or just adds noise (and risk) to the context window.

Skills, proven — that's the standard SkillzBench is building toward, one verified package at a time.

From Framework to Practice: Curating What Your Agent Sees

If you can't tell which tool your agent should use in a given situation, don't expect the model to figure it out. That's the blunt principle Elastic's search-labs team applies to tool curation: if a human engineer can't definitively say which tool fits a situation, an AI agent can't do better. The practical takeaway is to curate the minimal viable set — every tool and skill you expose to an agent should earn its place in the context window.

This turns context engineering from theory into a working checklist:

• Audit what your agent actually sees. List every tool, skill, and document in its context, then cut anything you can't justify.
• Prefer just-in-time loading. Following Anthropic's Agent Skills pattern, keep lightweight references in context and load full content only when needed, rather than pre-stuffing everything.
• Apply RAG to tool selection. LangChain found that retrieval over tool descriptions improved tool selection accuracy threefold — a strong argument for selecting tools dynamically instead of loading them all.
• Treat every piece of context as a potential attack surface.

That last point deserves emphasis. Prompt injection ranks first on OWASP's 2025 Top 10 for LLM Applications, appearing in 73% of production AI deployments — yet only 34.7% of organizations have dedicated defenses. The risk compounds with skills: published research found roughly 26% of published agent skills carry at least one vulnerability. When a skill's instructions enter your agent's context, so does everything wrong with it.

This is the gap SkillzBench is building against. As AIQ Labs' independent testing lab for agent skills, it's developing infrastructure for exactly this curation problem: version-pinned verification against commit SHAs, security scanning and license checks on every skill, sandboxed benchmarking that measures each skill's Lift in percentage points with and without it, and per-agent loadouts delivered through a single MCP server — so an agent only sees the skills and tools that were deliberately chosen for it.

The discipline is the same whether you're managing three tools or three hundred: curate deliberately, verify continuously, and measure what each addition actually contributes. Context rot is real — accuracy drops 24.2% when relevant information is buried in longer contexts — so every unvetted skill or unused tool isn't just a security risk, it's a performance tax. The teams that treat context as engineered infrastructure, rather than an accumulating pile, are the ones whose agents stay reliable in production.

Join the SkillzBench waitlist to get early access when the lab opens.

Getting Started: Your First 30 Days of Context Engineering

Knowing that the model isn't the bottleneck — the context it receives is — changes how you spend your first month with agents. Andrej Karpathy's mental model is the right starting point: the LLM is like a CPU, and its context window is like RAM, a finite working memory you must manage deliberately. Here's a 30-day plan to do exactly that.

Week 1: Audit your context window like RAM. Log what actually enters your agent's context across real sessions. Look for the four failure modes — poisoning, distraction, confusion, and clash — that Drew Breunig's framework, cited by LangChain, identifies as the ways long-context agents degrade. Remember that accuracy drops 24.2% when relevant information sits buried in longer contexts, per research cited by Glean.

Week 2: Kill the prompting tax. If you're manually rewriting business context into every prompt, every session, you're paying what Atlan calls the "prompting tax" — a data governance problem expressed as a prompting problem. Manual prompting stays manageable only at roughly 25–50 prompts; governed, metadata-driven context engineering scales past 200 without debt, as one e-commerce case study demonstrated in under six months.

Weeks 3–4: Structure and govern your assets. Adopt structured outputs — an underrated practice many AI developers skip, according to Prompting Guide AI — and stop treating skills and tools as copy-paste installs. Curate the minimal viable set of tools, because if a human can't say which tool applies, the agent can't either, as Elastic argues.

Your first-month checklist:

• Map every input entering your agent's context window and flag duplicates, stale data, and contradictions.
• Move repeated context into governed, reusable assets instead of retyping it per session.
• Define structured output schemas for every agent-to-system handoff.
• Inventory your installed skills and tools, and assign each a trust level rather than trusting by default.

That last point matters more than it sounds. Research has found roughly 26% of agent skills carry at least one vulnerability, and prompt injection sits at #1 on OWASP's 2025 Top 10 for LLM applications. Skills are supply-chain dependencies, not snippets — they deserve version pinning, license checks, and security scans before they touch production.

This is why SkillzBench exists. We're building an independent testing lab that verifies, benchmarks, and equips agent skills — pinning every version to a commit SHA, testing each skill in a sandbox with and without it, and publishing the measured Lift alongside a 0–100 trust score. Instead of copy-pasting an unverified SKILL.md, you'll get loadout-ready skills with evidence behind them.

Join the SkillzBench pre-launch waitlist to be first in line when verified, benchmarked skills go live. Skills, proven — before your agents depend on them.