Wei, Wang, et al. (2022) defined a chain of thought as a series of intermediate natural-language reasoning steps, and showed that putting such steps in the examples of a few-shot prompt is enough for the model to imitate them:
Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now? A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.
With just eight such examples, PaLM (540 billion parameters) achieved the then state of the art on GSM8K, a set of grade-school math problems. The effect depended on scale: in models below about 100 billion parameters, the chains were fluent but illogical and hurt performance (2022, sec. 3.2). Soon after, it turned out that adding "Let's think step by step" was enough to elicit chains without examples (Kojima et al., 2022; Buckner, 2024, pp. 279–280).
Why does it work? One explanation is computational: each generated token is one more pass through the network, so writing out the steps gives the model more computation per problem and lets it lean on partial results already written (Alammar & Grootendorst, 2024, pp. 185–187); hence its link to test-time compute and to reasoning models. Alammar & Grootendorst (2024) compare it to Kahneman's "System 2" (2024, p. 185).
To a logician, the chain looks like a derivation, but it is not one. Shanahan (2024) puts it precisely: if the model reasons correctly, it does so by mimicking well-formed arguments from its training data or the prompt. A theorem prover is trustworthy because its causal process mirrors the truth-preserving structure of inference; a model that merely mimics offers no such guarantee (2024, p. 76). Wei, Wang, et al. (2022) themselves leave open whether the network is actually "reasoning" (2022, sec. 6). And there is a faithfulness problem: nothing ensures that the written chain is the cause of the answer rather than an after-the-fact rationalization.