An encoder takes an input sequence x₁…xₙ and returns continuous representations z₁…zₙ (Vaswani et al., 2017, sec. 3): one vector per position, or a single one summarizing the whole. It does not produce text; it produces embeddings, whose value lies in what they make possible downstream.
History. In the first neural machine translation networks, the encoder was a recurrent network that read the sentence word by word and handed over its last hidden state as a summary: "I love llamas" → vector → "Ik hou van lama's" (Alammar & Grootendorst, 2024, pp. 11–13). In the Transformer, the encoder is a stack of six blocks of self-attention and feedforward (MLP) block layers in which every word can look at every other, backward and forward (Vaswani et al., 2017, sec. 3.1; Alammar & Grootendorst, 2024, p. 16).
Encoder-only models. Keeping only the encoder gave rise to "representation models": BERT and Sentence-BERT, used to classify, cluster and search, but not to write (Alammar & Grootendorst, 2024, pp. 18–20). The image encoder of Vision Transformer (ViT) and the two towers (text and image) of CLIP are the same idea in other modalities; the encoder of a VAE maps each image to a region of a latent space. ELMo was a bidirectional recurrent encoder.
What "encoding" does not mean. It is not enciphering with a key, nor translating into a code fixed in advance. The code is learned: nobody decides how many dimensions it will have or what each will mean. So the interesting question is not what a vector "says" (it says nothing), but what information can be recovered from it, and with what guarantees; that is the job of probing.