How can I improve my AI prompts?

Why Your AI Prompts Keep Underperforming

You rewrote the prompt three times, the model still missed the point, and now you're wondering whether the tool is broken — or whether you are. Neither. The real issue is that prompting is a genuine skill gap, and it's blocking AI adoption across entire teams.

IDEO's Savannah Kunovsky frames the problem bluntly: using LLMs requires knowing how to talk to a computer rather than a person, and she identifies that friction as an actual barrier to adoption. You're not chatting with a colleague who fills in gaps from shared context. You're issuing instructions to a system that takes your words literally, at scale, every single time.

The stakes are higher than most people assume. According to market research on prompt engineering, inadequately crafted prompts may result in irrelevant, biased, or inaccurate outputs. That's not a minor annoyance — it's the difference between an AI assistant that earns trust and one that quietly erodes it. IBM goes so far as to call prompt engineering "the new coding", the key to unlocking AI tools' full potential.

Then there's the problem that catches almost everyone off guard: there are no universal prompting standards. A prompt that yields effective results for GPT-4 may not perform well in Gemini or Llama, so engineers must continuously test different wording, formatting, and logical structures per model. Worse, prompts degrade over time — structures that worked yesterday lose effectiveness as models evolve. Your best prompt has an expiration date, even if nobody tells you what it is.

So when your outputs keep disappointing, the causes usually cluster into three patterns:

• Conversational framing — writing as if the model shares your context, instead of specifying exactly what to do and how.
• Unspecified quality — vague requests that leave the model guessing about format, tone, and scope, producing irrelevant or biased results.
• Model drift — prompts tuned for one model or an older version, never re-tested as systems change.

And sometimes the prompt was never the problem at all. As agent optimization practitioners warn, an agent failure is often a retrieval, tool, or infrastructure failure that shows up as a bad answer. If you can't tell the difference, you'll spend a week tuning a prompt against a bug.

The good news: these failures are diagnosable and fixable. If you're building agents, the same discipline applies to the skills and tools you attach to them — SkillzBench is building free benchmarking and security scanning so you can see which agent skills actually lift performance before you deploy them.

The Fundamentals: Iterate, Be Specific, and Say What TO Do

If your AI outputs feel vague or off-target, the fix usually isn't a clever trick — it's going back to the fundamentals. Prompting is fundamentally an iterative process that rewards experimentation, not one-shot perfection, according to widely referenced prompting guidance built on OpenAI best practices.

Start simple, then add detail incrementally. There are no magic tokens or keywords that guarantee better results — what matters is a good format and a descriptive prompt. That means leading with clear instructions, using direct action verbs like "Write," "Classify," or "Summarize," and separating instructions from context with markers like "###". Providing an example of your desired output format also helps the model understand exactly what you want.

Perhaps the most counterintuitive rule: say what TO do, not what NOT to do. The prompting guide illustrates this with a movie-recommendation chatbot: a prompt built around "DO NOT ASK FOR INTERESTS" failed, while the same instruction reframed positively — telling the model what to do instead, with a fallback response — worked. Positive framing encourages specificity; prohibitions leave the model guessing.

A practical checklist to apply right away:

• Begin with a minimal prompt, then add elements one at a time and observe how output changes.
• Place instructions first and use separators like "###" to distinguish instructions from context.
• Show the model an example of the output format you expect, not just a description of it.
• Replace every "don't do X" with "do Y instead," including a fallback behavior for edge cases.
• Keep prompts specific, simple, and concise — clarity beats cleverness.

These basics are corroborated beyond tutorials: a peer-reviewed review in Cell Press Patterns notes that designing clear, concise prompts is becoming an essential competency, and that foundational techniques pair well with advanced methods like chain-of-thought and self-consistency, which can significantly enhance model performance (Patterns review). Structure pays off commercially, too — one analysis of production system prompts found that shorter, structured prompts cut serving costs by 76% while reducing output variance and latency (prompt engineering analysis).

One caveat worth remembering: prompts don't behave identically across models, so wording and structure that work well in one system may underperform in another. Treat your fundamentals as a starting point, verify results per model, and iterate from real outputs rather than assumptions. And if you're building agents rather than chat prompts, the same principle applies to the skills you attach — SkillzBench is building free benchmarking and trust scores so you can see which agent skills genuinely lift performance before you rely on them.

Level Up: Chain-of-Thought, Structured Inputs, and Shorter Prompts

When basic prompting stops moving the needle, the techniques that peer-reviewed research says "significantly enhance the performance of models" come into play: chain-of-thought, self-consistency, and generated knowledge prompting. The market agrees — chain-of-thought is the fastest-growing technique segment, while n-shot prompting still holds the largest share at 40% in 2025, according to Precedence Research.

These advanced methods shine on harder tasks. Chain-of-thought asks the model to reason step by step before answering. Self-consistency runs that reasoning multiple times and takes the majority answer. Generated knowledge prompting has the model first produce relevant facts, then answer using them — a trio named in a Cell Press Patterns review as reliably boosting performance.

Structured context is the other half of the equation. IBM recommends leveraging RAG, summarization, and structured inputs like JSON to guide models toward more accurate responses. For agent builders, this matters doubly: everything flowing through MCP — inputs and outputs — consumes the model's context window, so budgeting context is part of good prompting.

Here's the counterintuitive part: shorter prompts often beat longer ones. One analysis compared two production system prompts across 100,000 daily calls and found the compact, structured version cost $706/day versus $3,000/day for the detailed one — a 76% cost reduction with less output variance and lower latency. The shorter prompt used a 212-token prompt with 400-token outputs instead of 2,500 tokens in and 1,500 out.

What makes prompts shorter and better:

• Bracketed, code-like structure with "always"/"never" lists instead of long prose
• If/then edge-case handling and explicit display instructions
• Detailed error handling derived from past testing failures
• Structured data formats like JSON rather than paragraphs of description

Both prompts analyzed came from products succeeding commercially — Cluely reached $6M ARR in two months, and Bolt hit $50M ARR in five months with its system prompt cited as one of the keys to its success. As practitioner Aakash Gupta puts it, "every instruction you write into a system prompt is a product decision."

One caveat: a prompt that performs well in one model may fail in another, so test wording and structure per model — and remember that agent failures often stem from retrieval or infrastructure bugs, not the prompt. When you're wiring advanced prompts into an agent, the skills you attach shape the context too; SkillzBench is building free benchmarking and security scanning for agent skills so you can see which ones genuinely lift performance. Join the waitlist to be first in line.

ctaText: Join the free SkillzBench waitlist socialProofText: Free, independent skill benchmarks and trust scores — money never touches the scores.

Test Before You Ship — and Diagnose Before You Tune

A prompt that works on a Tuesday demo can quietly break in production — and the teams that catch it before users do are the ones treating prompts like code, not like clever sentences. That means testing changes against real data, and knowing when the prompt isn't actually the problem.

The most expensive mistake in prompt engineering is rewriting instructions that were never broken. As Pydantic's analysis of agent optimization puts it, "an agent failure is often a retrieval, database, tool, or infrastructure failure that shows up as a bad answer." Platforms that only see LLM spans cannot tell you when the prompt was never the issue.

The practical discipline: trace the full stack first. Did the retrieval layer return the right documents? Did the tool call succeed? Only after ruling those out should you touch the prompt — otherwise you risk, in Pydantic's words, "tuning a prompt against an infrastructure bug for a week."

Once the prompt is genuinely the suspect, evaluate it systematically. production-focused guidance recommends three practices:

• Build test datasets from production traces and edge cases — not from invented examples that flatter your prompt.
• Run automated evaluations combining rule-based checks with LLM-as-judge scorers, using the same scorers in testing and production.
• Version every change and validate it against measurable metrics before deployment — some teams even block merges when prompts fail quality thresholds.

The stakes are real. A single prompt change can cause a chatbot to hallucinate product details, an agent to select the wrong tool, or a system to fabricate citations — failures users see immediately. And because prompts perform differently across models, with a prompt that works for GPT-4 potentially failing in Gemini or Llama, per-model testing is mandatory, not optional, according to market research on the prompt engineering field.

If your agent uses external skills or tools, the same logic applies: benchmark with and without the component, using the same model, so you can see the actual lift rather than trusting a README. That's exactly the approach SkillzBench is building — sandboxed per-skill benchmarking, security scanning, and trust scores for agent skills, plus a free MCP connector, so you can verify what genuinely helps before it ships. Versioning and testing every prompt change against real data catches issues during development, before users encounter them — and that discipline, more than any clever phrasing, is what separates reliable prompts from lucky ones.

Beyond the Prompt: Context, Skills, and What Comes Next

A perfect prompt won't save a broken agent. As practitioner analysis bluntly puts it, "an agent failure is often a retrieval, database, tool, or infrastructure failure that shows up as a bad answer" — and tuning a prompt against an infrastructure bug wastes a week solving the wrong problem.

That's why experienced builders think bigger than the prompt itself. Aakash Gupta frames prompt engineering as a subset of context engineering, which he says "can do 85% of the heavy lifting" in many cases. IBM makes a similar point: crafting better prompts is only the beginning, and techniques like RAG, summarization, and structured inputs such as JSON do the real work of guiding models toward accurate responses (IBM's guide).

For agent builders, the practical frontier is tooling. The Model Context Protocol, introduced by Anthropic in November 2024 and since adopted by OpenAI, Microsoft, and Google, standardizes how models pull in external tools and data. But InfoQ's reporting offers a critical caveat: MCP's utility "might be somewhat limited by the context windows of the models because everything that happens with MCP — the input, the output — goes into that model context." Every integration you connect consumes budget, so manage it deliberately — one analysis found a 76% cost reduction simply from moving to shorter, structured prompts.

The rest of the workflow — browsing, verifying, testing, and maintaining tools — is increasingly being automated by agent skills, and that's where the ecosystem gets messy:

• Skills are everywhere, but quality varies wildly, and published research found roughly 26% carry at least one vulnerability.
• No universal prompting standards exist — prompts perform differently across GPT-4, Gemini, and Llama, so anything layered on top needs per-model testing too.
• Prompts and tools degrade as models evolve, making continuous maintenance — not one-time setup — the real job.

SkillzBench (skillzbench.com) is being built to close exactly this gap: a free, independent directory of agent skills plus a free MCP connector, with per-skill benchmarking that tests each skill in a sandbox with the same model, with and without the skill. It's also building security scanning, license checks, and trust scores — including a published Lift metric showing how many percentage points a skill actually adds — so you can equip agents with skills that demonstrably lift performance rather than ones that merely look good in a README.

The takeaway for anyone serious about prompting: your prompt is a product decision, not a text file. Ground it in the right context, budget that context carefully, verify your tools, and measure everything.