A sequence-to-sequence task takes a string of symbols and returns another, with no one-to-one correspondence between positions: “I love llamas” (three words) becomes the Dutch “Ik hou van lama's” (four) (Alammar & Grootendorst, 2024, p. 11). Translation, summarization, question answering, grammatical error correction and even syntactic parsing written out as a string all fit this mold. A linguist will recognize the notion of transduction: a relation between two languages computed by a machine.
The architecture that made it tractable for neural networks is the encoder–decoder: one network reads the input and summarizes it in a context vector; another generates the output word by word, autoregressively (Sutskever et al., 2014; Alammar & Grootendorst, 2024, pp. 11–12). This is the Seq2seq species. Its weak point was the bottleneck: the entire content of the sentence had to fit into a single vector, and long sentences degraded. Attention solved this by letting the decoder consult all the encoder's states (Bahdanau attention) (Bahdanau et al., 2015; Buckner, 2024, pp. 269–270), and the Transformer kept attention and dropped recurrence.
Tiny example. Input: “summarize: The board met on Monday and, after five hours of debate, approved the budget.” Output: “Budget approved.” Neither the length nor the order of the output is fixed by the input. T5 took the idea to its limit: every task, whether classifying, translating or scoring similarity, is rewritten as text in and text out, with the task's name as a prefix (Raffel et al., 2020).
In a broad sense, a chat LLM is also a sequence-to-sequence machine: it takes in some text and tries to complete it (Alammar & Grootendorst, 2024, p. 22). The difference lies in the architecture, not the task: GPT-style models do everything with a single decoder that treats input and output as one sequence.