Chain-of-thought prompting
Chain-of-thought prompting is a technique for coaxing step-by-step reasoning out of a Large language model simply by asking, or showing, it to "think out loud" before giving a final answer. Rather than jumping straight from question to answer, the model is guided — through examples or instructions in its prompt — to generate a visible sequence of intermediate reasoning steps, much like a student showing their work on a math test.
The technique was introduced in a landmark 2022 paper by Wei and colleagues, which found that a related zero-shot approach is to simply append "let's think step-by-step" to the prompt, without any few-shot exemplars, a variant later formalized by Kojima et al. as "Large Language Models are Zero-Shot Reasoners." The original, few-shot version instead embeds a handful of worked examples directly in the prompt, each demonstrating the desired reasoning chain before its answer. Wei et al. (2022b) explores how chain-of-thought (CoT) prompting improves the ability of LLMs to perform complex reasoning such as arithmetic reasoning, commonsense reasoning, and symbolic reasoning, and by guiding LLMs through generating intermediate reasoning chains prior to reaching a final solution, CoT prompting has propelled the multi-step reasoning abilities of LLMs to an extraordinary level.
Curiously, the effect appears to be an emergent property: it barely helps small models, but blossoms once a Transformer-based model — descendants of the architecture from Attention Is All You Need — crosses a certain scale threshold, as seen in models like GPT-3 and its successors in the broader GPT family. Some researchers frame this as the model approximating slower, deliberate "System 2" cognition rather than instantaneous pattern-matching, though the mechanism by which CoT yields improvement is currently not fully understood.
Chain-of-thought prompting sits alongside other reasoning-boosting tricks in the prompt-engineering toolkit, and is often combined with Retrieval-augmented generation to fetch facts mid-chain, or paired with Reinforcement learning from human feedback and instruction-tuning approaches pioneered by InstructGPT to make models more willing to reason transparently by default. It contrasts with earlier sequence architectures like Long short-term memory and classic Sequence to sequence learning models, which lacked the Self-attention and Attention mechanisms that let modern transformers attend flexibly across a long reasoning trace. Encoder-decoder models such as T5 and bidirectional models like BERT predate the technique's popularization, having been built before chain-of-thought became a standard evaluation lens.
Beyond raw accuracy gains, chain-of-thought outputs offer a kind of interpretability window — a peek into the model's "process" — though skeptics note the stated reasoning may not always reflect what the network actually computed internally. Variants now include self-consistency sampling, tree-structured reasoning, and hybrids with Agent-based modeling-style multi-step planning, extending the idea into full Networked systems of cooperating model calls.