How do I make ChatGPT think longer?

Why ChatGPT's Short Answers Fail on Hard Problems

You know the feeling: you hand ChatGPT a genuinely hard problem — a gnarly proof, a multi-step business plan, a stubborn bug — and it fires back a confident, shallow answer in seconds. The model isn't lazy. It's optimizing for speed, and on complex tasks, that trade-off quietly costs you accuracy.

The hidden mechanism is something called reasoning tokens. Reasoning models generate long internal traces — sometimes thousands of tokens — that users never see but that "significantly improve final answer quality," according to engineering analysis of reasoning LLMs. When ChatGPT answers quickly, it's spending few of those tokens. When it thinks longer, it spends more.

This is the principle researchers call inference-time scaling: spending more compute after training, at answer-generation time. It works — up to a point. The evidence is striking:

• o1's AIME math score climbed from 74% with a single sample to 93% with re-ranking of 1,000 samples, while GPT-4o managed only 12%.
• o3 jumped from 75.7% to 87.5% on ARC-AGI as compute increased — past the 85% human-level mark.
• GSM-8K accuracy rose from 82.2% to 87.3% as average thinking tokens grew from 385 to 1,100, per a 2025 arXiv study.

But there's a trade-off: latency, cost, and accuracy pull against each other. Reasoning models take "seconds to minutes longer" than standard models, and extended thinking costs real money — some estimates put o3's per-token cost at 15x GPT-4's. As researcher Sebastian Raschka notes, extreme inference scaling is "totally worth it" only where accuracy matters more than speed or cost.

That's why longer thinking is a deliberate lever, not a hack. OpenAI itself reports that "if we let it think longer, its performance keeps climbing," and builds pro-tier models explicitly designed to think longer. The catch: you should pull that lever selectively, on problems that deserve it.

There's a caveat worth knowing before you max out the setting. The same arXiv research found that forcing extended thinking produces an "overthinking" pattern — accuracy rises, then declines — and that verbose reasoning is largely superfluous outside math-heavy tasks. Longer is not automatically better; directed longer is.

For those building agent workflows, reasoning depth also extends through tools and skills — curated instruction packages can steer how long and how well an agent thinks. SkillzBench, currently in pre-launch, is building a free directory and MCP connector to discover, verify, and benchmark exactly those agent skills.

The Official Levers: Reasoning Models and Effort Settings

The most reliable way to make ChatGPT think longer isn't a clever prompt — it's a dial OpenAI built into the product itself. Since the o-series launched, "thinking longer" has been an official, engineered capability, not a workaround.

Pro-tier reasoning models are explicitly designed for extended thinking. OpenAI describes its pro models as "designed to think longer and provide the most reliable responses," and the company has stated that "if we let it think longer, its performance keeps climbing" (OpenAI's announcement). Under the hood, these models generate long internal "reasoning tokens" — sometimes thousands of them — that users never see but that measurably improve answer quality (according to WorkOS).

The second lever is the user-selectable reasoning-effort setting. Depending on the model and interface, you can choose low, medium, or high effort in the API, or pick "high" variants in ChatGPT, and OpenAI evaluates its models at high effort for a reason (per Helicone's analysis).

The evidence that more compute translates into better scores is striking:

• o3 on ARC-AGI jumped from 75.7% at low compute to 87.5% at high compute — clearing the 85% human-level baseline.
• o1 on AIME scored 74% with a single sample, 83% with 64-sample consensus, and 93% with re-ranking of 1,000 samples — while GPT-4o managed just 12%.
• o3-mini-high scored 96.7% on AIME versus o1's 83.3%.

This is what researchers call inference-time scaling: spending more time and money after training, when the model generates its answer. Sebastian Raschka notes it's a trade-off between latency, cost, and accuracy — but "in certain applications, where accuracy matters more than latency and cost, extreme inference-scaling can totally be worth it" (Ahead of AI).

Tool use compounds the effect. o3 and o4-mini can chain tools like Python and web search inside their reasoning — o4-mini hit 99.5% pass@1 on AIME 2025 with a Python interpreter, and o3 once reasoned for 55 seconds on a hard math problem while interleaving code execution.

The practical rule: be selective. Reserve pro models and high effort for complex math, coding, and multi-step planning where accuracy justifies the cost and the seconds-to-minutes of extra latency. For simple tasks, extended thinking is "both inefficient and expensive."

If you're extending reasoning into agentic workflows — giving models tools, skills, and multi-step autonomy — SkillzBench is building a free directory and MCP connector to discover, verify, and benchmark agent skills, so the capabilities you bolt on are proven, not guessed. Join the waitlist at skillzbench.com: skills, proven.

Prompting Techniques: Chain-of-Thought and Tool Use

Before spending money on bigger models, try two sentences: "Let's think step by step." Zero-shot chain-of-thought is the simplest prompting technique for eliciting step-by-step reasoning without any examples, and it has been shown to flip incorrect answers into correct ones on reasoning tasks, according to the Prompting Guide.

For more control, few-shot chain-of-thought adds one or more worked examples to the prompt. Even a single demonstration of the reasoning style you want can guide the model to reproduce that pattern on your problem. The trade-off is cost: chain-of-thought prompting can improve performance by up to 80% on certain tasks, but response length and cost rise proportionally, so reserve it for problems where accuracy genuinely matters.

There is also a useful counterpoint: verbosity is not the same as depth. A study on Concise Chain-of-Thought found that adding "Be concise" to prompts cut response length by nearly half with negligible accuracy impact overall — except in math, where GPT-3.5 suffered a significant 27.69% accuracy penalty under concise reasoning. In other words, long-winded explanations add little outside quantitative tasks, and verbose CoT conventions in prompt engineering rest largely on anecdotal rather than empirical evidence.

The bigger leap comes from giving the model tools to use inside its reasoning chain. OpenAI's o3 and o4-mini can agentically chain tools — web search, Python, image manipulation — within their own thinking, typically finishing tasks in under a minute. The results are striking: o4-mini reached 99.5% pass@1 on AIME 2025 with a Python interpreter, and in one example o3 reasoned for 55 seconds on a hard math problem while making Python calls mid-thought, per OpenAI's announcement.

For teams building agent workflows, this tool-augmented reasoning is where skills and connectors matter. SkillzBench is building a free directory of agent skills plus a free MCP connector, so you can discover, verify, and benchmark the skills your agents use to extend their reasoning in exactly this way.

A few practical rules of thumb:

• Start with zero-shot CoT — "Let's think step by step" — before paying for anything heavier.
• Add a single worked example when you need the model to follow a specific reasoning pattern.
• Enable tool use (Python, web search) for math, coding, and research-heavy tasks, where it delivers measurable gains.
• Skip verbose-reasoning instructions on non-math tasks; concise prompts save cost without hurting accuracy.

Prompting is the cheapest lever you have for making ChatGPT think longer — and combining it with tool access turns a single reasoning chain into an agentic workflow.

The Overthinking Trap: When Longer Thinking Makes Answers Worse

Here's a twist that undermines everything we've assumed so far: in some experiments, making a model think longer actually made it less accurate. The "more tokens, better answers" intuition doesn't hold up under scrutiny.

A 2025 study from researchers at the University of Maryland, Princeton, and Amazon AGI found that extended thinking follows a non-monotonic pattern — accuracy rises at first, then declines as the model keeps going. The authors describe this as "overthinking" and conclude that test-time scaling through extended thinking is not an effective way to use a thinking budget. Their GSM-8K results show the pattern clearly: accuracy climbed from 82.2% to 87.3% as thinking tokens grew from 385 to 1,100, but the gains didn't continue indefinitely. Additional thinking, they found, increases output variance — creating "an illusion of improved reasoning while ultimately undermining precision."

The same research offers a better use of the same token budget. Instead of one long reasoning trace, the researchers tested parallel thinking — running multiple independent reasoning paths and taking the majority answer. That approach achieved up to 20% higher accuracy than extended single-trace thinking, and 22% higher under a 16K token budget. In practical terms: ask the same hard question several times and compare answers, rather than forcing one extremely long chain of thought.

Verbosity itself turns out to be largely superfluous, too. The Concise Chain-of-Thought (CCoT) study added "Be concise" to chain-of-thought prompts and cut average response length by 48.7% with negligible accuracy impact overall — GPT-4 accuracy actually ticked up 0.25%. The paper notes that verbose reasoning conventions in prompt engineering rest on "anecdotal rather than empirical evidence."

There's one important exception: math. GPT-3.5 suffered a statistically significant 27.69% accuracy penalty on math problems when forced to reason concisely. So the working rule looks like this:

• For math and hard multi-step problems, longer and more detailed reasoning genuinely helps — don't cut it short.
• For everything else, "be concise" is a nearly free win: half the tokens, roughly half the cost, no meaningful accuracy loss.
• When a question really matters, consider multiple shorter attempts with majority voting instead of one marathon trace.
• Treat "longer thinking = better" as a hypothesis to test, not a law.

This selectivity principle matters for anyone building agent workflows, too. The tools and skills an agent can call during reasoning — code execution, search, structured data lookups — often extend its effective thinking more reliably than raw token count does. SkillzBench is building a free directory and free MCP connector to help you discover, verify, and benchmark agent skills for exactly this purpose, so your agents spend their thinking budget where it actually pays off.

Your Playbook: When to Pay for Deep Thinking — and How Skills Extend It

Paying for extended reasoning is like paying for express shipping: worth it when the contents are valuable, wasteful when they're not. The research is clear that deep thinking helps — but only in the right situations, and only up to a point.

Start with the official levers. OpenAI designs its pro-tier models "to think longer and provide the most reliable responses," and reasoning-effort settings (low, medium, high) are user-selectable in ChatGPT and the API, per OpenAI's announcement and pricing analyses. The evidence for where this pays off is strong: o3 jumped from 75.7% to 87.5% on ARC-AGI at high compute, and o4-mini hit 99.5% on AIME 2025 when allowed to chain Python tool calls inside its reasoning.

Your decision framework:

• Pay for high effort on math, coding, and planning. These are the domains where accuracy climbs with compute and where concise reasoning actually hurts — one study found a 27.69% accuracy penalty on math under "be concise" prompting.
• Use parallel thinking for critical answers. A 2025 arXiv study found that splitting the same token budget across multiple independent reasoning paths with majority voting beats one long trace by up to 20% — ask the model several times and compare, rather than forcing one marathon chain.
• Skip extended thinking on simple tasks. Summarization and translation gain little; as WorkOS notes, reasoning models on easy tasks are "both inefficient and expensive."

Watch for the overthinking cliff. The same arXiv research found accuracy follows a non-monotonic curve — it rises, then declines as the model second-guesses itself. If a high-effort answer seems worse than a medium-effort one, that's not your imagination.

Finally, tools extend reasoning further than tokens alone. OpenAI's benchmarks show agentic tool use inside the reasoning loop dramatically improves results, and curated agent skills (SKILL.md instruction packages for coding agents and chat apps) push this even further. SkillzBench is building a free directory that discovers, verifies, and benchmarks skills — pinning every version, security-scanning it, and testing it in a sandbox to publish the measured Lift — plus a free MCP connector so your agent only sees the tools you choose. Given research finding roughly 26% of published skills carry at least one vulnerability, verified skills are the safer way to make your agent think harder about the right things.