In cross-attention there are two sequences. One (Y) issues the queries; the other (X) supplies keys and values:
CrossAttention(Y, X) = softmax((Y·W_Q)(X·W_K)ᵀ / √dₖ) · (X·W_V)
In the original Transformer, every decoder block has such a layer: its positions (the target text) query the output of the encoder (the source text), so that each word being written can look at the whole source sentence (Vaswani et al., 2017, sec. 3.2.3; Alammar & Grootendorst, 2024, pp. 16–17). That is the difference from self-attention, where all three come from the same sequence.
Genealogy. Bahdanau attention's attention was already cross-attention: when generating each French word, the decoder weighted the English words (Bahdanau et al., 2015, sec. 3.1). Its weight maps work as soft alignments. To translate the European Economic Area as la zone économique européenne, the model aligns zone with Area, skipping two words, and then steps back one word at a time to complete the phrase: adjective–noun order is reversed (Bahdanau et al., 2015, sec. 5.2.1).
Where it lives today. In encoder–decoder models such as T5, and as a bridge between modalities: in latent diffusion, cross-attention layers let the image under construction "consult" the text prompt (Rombach et al., 2022) (Diffusion model). Decoder-only models such as GPT-3 lack it: everything (instruction, document, answer) goes into a single sequence and is handled by self-attention.
A philosophical angle. Cross-attention is the place in the architecture where one representation gets connected to another: a word to its translation, an image region to a phrase. It is tempting to read those weights as reference or grounding (see symbol grounding problem). But they are learned correlations, and soft ones: unlike the "hard" alignments of statistical translation, they do not force each word to pair with another, nor send words without a counterpart "nowhere" (Bahdanau et al., 2015, sec. 5.2.1). That two things are aligned does not yet say what either of them is about.