What is chain of thought prompting?
Chain of thought prompting means asking a model to reason through a problem before it answers. Instead of going straight to a conclusion, the model walks through intermediate steps, and those steps become part of the output you read and can check.
The term comes from a 2022 Google Brain paper by Wei et al., which showed that large language models produce more accurate answers on reasoning tasks when prompted to think through each step explicitly. The finding was notable because the improvement required no retraining. A different prompt was enough.
For a team lead, the practical implication is straightforward. When a model fails at a multi-step task, the usual fix is not a better model. It is a better prompt. Telling the model to show its work gives it (and you) a path to follow, and it gives you somewhere to spot where things went wrong.
This is distinct from simply asking for a longer answer. Chain of thought prompting structures the reasoning, not just the length. A prompt that says "explain your thinking as you go" will behave differently from one that says "give me a detailed answer."
It sits within the broader family of prompting techniques that shape how a model approaches a task, rather than just what it is asked to produce.
How it works
The core mechanic is simple. Instead of asking a model for an answer, you ask it to show its work on the way to an answer. The model generates a sequence of intermediate steps, and each step conditions what comes next. Errors that would be invisible in a one-shot answer tend to surface in the chain, where you can spot and correct them.
There are two main ways to trigger this behavior.
The first is a verbal instruction. Adding a phrase like "think step by step" or "walk through your reasoning before giving a final answer" is often enough. This approach, sometimes called zero-shot chain of thought, requires no examples. It works reasonably well on straightforward problems and is the fastest thing to try.
The second is few-shot chain of thought. You provide one or more examples that demonstrate the reasoning pattern you want, then present the real question. The model matches the pattern. This takes more setup but tends to produce more consistent results on complex or domain-specific problems.
Concretely, when a model is forced to name each step, it cannot skip from premise to conclusion without passing through the logic in between. That constraint is where the accuracy gains come from.
Worked examples
The three prompts below move from a simple decision to a more complex analytical task. Each shows the full prompt, the model's response, and a note on what the reasoning step changed.
Example 1: Choosing between two vendors
Prompt:
We need to choose between two project management tools. Tool A costs $12/user/month,
has offline access, and our team already knows it. Tool B costs $8/user/month,
is cloud-only, and has better reporting. We have 40 users and spotty office Wi-Fi.
Think through the trade-offs step by step before giving a recommendation.Model output: [NEEDS REAL OUTPUT]
Commentary: Adding "think through the trade-offs step by step" forces the model to surface the Wi-Fi constraint before landing on a number. Without it, the cheaper price tends to win by default.
Example 2: Root-causing a customer complaint pattern
Prompt:
Our support tickets spiked 40% in the second week of March. Think through the
possible causes step by step, ruling each one in or out, before naming the
most likely root cause.Model output: [NEEDS REAL OUTPUT]
Commentary: The instruction to rule causes in or out produces a structured elimination. A direct "what caused this?" prompt typically returns a single guess with no reasoning shown.
Example 3: Drafting a difficult message
Prompt:
I need to tell a long-standing client that their project will be delayed by three
weeks due to a resourcing problem on our side. Work through how the client is
likely to feel, what their main concerns will be, and what we owe them, then
draft the email.Model output: [NEEDS REAL OUTPUT]
Commentary: Walking through the client's perspective first shifts the tone of the draft. Prompts that jump straight to "write the email" tend to produce formal but hollow apologies.
When to use it and when not to
Chain of thought prompting earns its keep on tasks where the answer depends on a sequence of decisions, not just recall. If getting from question to answer requires holding multiple conditions in mind at once, asking the model to show its reasoning almost always improves accuracy.
Good fits include:
- Multi-step math or financial calculations
- Legal or policy questions where several rules interact
- Diagnosing a process problem (why is the churn rate up this quarter?)
- Comparing options against a set of weighted criteria
- Drafting recommendations that need to cite specific evidence
There are real situations where it is the wrong call. Factual lookups, short summarization tasks, and single-step classifications do not benefit from visible reasoning. Asking a model to "think step by step" before answering "What is the capital of France?" adds nothing except length. On models billed by token, that length has a cost.
It is also the wrong tool when you need a tight, polished output, such as a final client email or a headline, and the reasoning would have to be stripped out anyway. In those cases, zero-shot prompting keeps things cleaner.
One more honest caveat: chain of thought does not fix a bad premise. If your initial question contains a faulty assumption, the model will reason from that assumption carefully and confidently to a wrong answer.
Chain of thought prompting compared with related techniques
Chain of thought prompting sits in a family of techniques that all try to improve output quality, but they solve different problems.
| Technique | What it does | Best for |
|---|---|---|
| Zero-shot prompting | Asks the model to answer with no examples or reasoning instructions | Simple, clear-cut tasks |
| Few-shot prompting | Provides examples of the desired output format | Tasks where format matters more than reasoning |
| Chain of thought prompting | Asks the model to show its reasoning before giving an answer | Multi-step problems where accuracy matters |
| System prompts | Sets persistent context, tone, or constraints for a conversation | Shaping behavior across an entire session |
The clearest distinction is between chain of thought and zero-shot prompting. Zero-shot works well when the task is straightforward: summarize this, translate that, classify this sentence. Chain of thought earns its keep when the answer depends on getting intermediate steps right.
System prompts are complementary rather than competing. You can embed a chain of thought instruction inside a system prompt so every query in a session follows that reasoning pattern automatically. That combination is particularly useful for teams who want consistent behavior without asking individuals to remember the technique each time.
Few-shot prompting and chain of thought can also be combined. You provide examples that themselves include reasoning steps, which gives the model both a format to follow and a reasoning style to replicate. The trade-off is a longer prompt and a higher token cost per request.
Common mistakes
The most common one is adding "think step by step" to a prompt that doesn't need it. Simple factual lookups, short rewrites, and straightforward classification tasks don't benefit from a reasoning chain. You just get a longer response with the same answer buried at the end.
The second mistake is being too vague about what the steps should cover. "Think through this carefully" is not a chain of thought instruction. It's an encouragement. A useful prompt tells the model which dimensions to reason across: assumptions first, then constraints, then options, then a recommendation. Vague prompts produce vague chains.
Third: trusting the chain uncritically. A model can produce a confident, well-structured set of steps that leads to a wrong conclusion. The intermediate reasoning is plausible-looking, not verified. For anything consequential, treat the chain as a draft argument to be checked, not a proof.
Finally, teams often copy a chain of thought prompt from one context and apply it to another without adjusting the reasoning structure. A prompt designed to evaluate a vendor contract will produce odd results when pointed at a hiring decision. The skeleton transfers; the content guidance usually doesn't.
Using this across a team
Solo use of chain of thought prompting is straightforward. Team use is where things get inconsistent fast. One person asks the model to think step by step; another skips that phrase entirely; a third adds it but buries it at the end of a long prompt where it does little work. The outputs diverge, and nobody knows why.
The fix is shared prompt templates. When the reasoning instruction is baked into a template rather than typed from memory, every team member starts from the same foundation. That matters most for recurring tasks: financial analysis, content briefs, customer escalation reviews, anything where quality consistency has real consequences.
A few practical steps that help when rolling this out to a group:
- Write the reasoning instruction into the template itself, not into onboarding documentation nobody reads twice.
- Include a worked example inside the template as a comment or placeholder, so new users see what a good output looks like before they run their first prompt.
- Review outputs periodically as a team. If one person's results are consistently stronger, look at their prompt structure. Often the difference is small and easy to copy.
Shared templates also make it easier to improve prompts over time. One person refines the chain of thought instruction based on real results, and everyone benefits immediately. That compounds in a way that individual prompt habits never do.
Frequently asked questions
What is chain of thought prompting?
Chain of thought prompting is a technique where you instruct an AI model to show its reasoning step by step before giving a final answer. Instead of asking for a direct response, you ask the model to think through the problem out loud. This tends to produce more accurate results on tasks that involve multiple steps, comparisons, or judgment calls.
Does chain of thought prompting work on all AI models?
It works on most large language models, including ChatGPT, Claude, Copilot, and Gemini, though the quality of the reasoning varies. Smaller or older models sometimes produce steps that look structured but contain logical errors, so the output still needs a human check. The technique generally performs better on models released after 2022.
When should I not use chain of thought prompting?
Skip it for simple, factual lookups where a direct answer is all you need. Asking a model to reason through "What is the capital of France?" wastes tokens and adds no accuracy. Save the technique for tasks that genuinely require multiple reasoning steps.
Does adding "think step by step" always help?
It helps often, but not always. On straightforward tasks it can introduce unnecessary verbosity, and on some creative tasks it can make the output feel mechanical. Treat it as a default for analytical work, not a universal rule.
Can I save chain of thought prompts for my whole team to use?
Yes. Storing tested prompts in a shared library means everyone starts from a consistent baseline rather than improvising each time. Convergence's prompt templates are designed for exactly this kind of reuse across tools like ChatGPT, Claude, Copilot, and Gemini.
Further reading
The original paper that introduced chain of thought prompting is Chain-of-Thought Prompting Elicits Reasoning in Large Language Models by Wei et al. (2022), published by Google Research. It is the primary source for the technique and worth skimming even if you skip the technical sections; the worked examples in the appendix are clear enough for any reader.
For a broader map of where chain of thought fits among other approaches, the Convergence guide to prompting techniques covers the full range, from basic instruction prompts through to more structured methods. If you are new to prompting in general, the piece on zero-shot prompting is a useful starting point before adding reasoning steps.
System prompts are worth reading alongside this page if you want to apply chain of thought at the team level. Encoding a reasoning instruction in a system prompt means every conversation in that context inherits it, which is a more reliable approach than trusting individual contributors to add it manually.
Finally, if you want ready-made prompts that already apply chain of thought structure, the Convergence prompt template library includes templates built for common team tasks, with the reasoning steps written in and tested across ChatGPT, Claude, Copilot, and Gemini.
Published: July 2025. Last updated: July 2025.