A decoder produces a sequence one element at a time and uses what it has already produced as input to the next step: it is autoregressive generation (Vaswani et al., 2017, sec. 3). In probabilistic terms, it decomposes the probability of an output into a chain of conditional probabilities:
p(y₁ … yₙ) = Πₜ p(yₜ | y₁ … yₜ₋₁)
(Bahdanau et al., 2015, sec. 2.1).
Three versions. (1) In recurrent sequence-to-sequence task networks, the decoder was a second recurrent network that started from the encoder's summary (Seq2seq). (2) In the Transformer, each decoder block has self-attention with a causal mask, a cross-attention layer over the encoder, and a feedforward (MLP) block network (Vaswani et al., 2017, sec. 3.1). (3) In "decoder-only" models the encoder and the cross-attention disappear: what remains is a stack of blocks with masked self-attention. That is what GPT-1 ("a 12-layer decoder-only transformer", (Radford et al., 2018, sec. 4.1)), GPT-3, LLaMA, ChatGPT (InstructGPT) and most current models are (Alammar & Grootendorst, 2024, pp. 20–21).
How it generates. At the top of the stack, a "language modeling head" turns the last vector into a score (logit) for every token in the vocabulary; a sampling strategy picks one; that token is appended to the text and everything starts again (Alammar & Grootendorst, 2024, pp. 75–79). The model itself only predicts; the loop that turns it into a writer is an external program.
An inherited name. In 1947 Warren Weaver imagined that a Russian text was "really written in English" but coded in strange symbols, waiting to be decoded (M. Mitchell, 2019, ch. 12). In a translator the metaphor still makes some sense: there is a source message. In a decoder-only model there is nothing to decipher; the name outlives the architecture it was coined for. That a system trained only to continue texts ends up answering questions is precisely what makes the debate about its understanding interesting.