After a convolution, the feature map says where each pattern was detected. Pooling summarizes it by zones: in 2×2 max pooling, each block of four values is replaced by the largest.
1 3 | 0 2
2 0 | 1 1 2×2 max pooling 3 2
----+---- ──────────────▶ 4 5
4 1 | 5 0
0 2 | 3 1
The map becomes four times smaller and, above all, less sensitive to exact position: with the maximum, a feature counts as present if it was detected anywhere in the zone (Witten et al., 2017, p. 438; Chollet, 2021, sec. 8.1.2). In training, the error is routed back only to the "winning" unit of each zone (Witten et al., 2017, p. 444).
Origin. It is the artificial counterpart of Hubel and Wiesel's "complex" cells, which respond to the same edge across nearby positions. Fukushima's Neocognitron averaged; today the maximum dominates (Buckner, 2024, pp. 124–126). It appears in Convolutional network (LeNet) and AlexNet.
Abstraction by subtraction. Buckner (2024) reads pooling as a form of abstraction in the tradition of Locke and Berkeley: it discards irrelevant variation (where the edge is, which font the letter is in) and keeps what matters for the category. Alternating with convolution, which composes, it lets the network learn which transformations to ignore (2024, pp. 127–129).
In language. The term returns with another sense: aggregating the vectors of all the tokens of a sentence into one. Sentence-BERT uses the mean of the output vectors by default (Reimers & Gurevych, 2019, sec. 3); BERT uses the vector of the special [CLS] token. Transformers do not usually pool between layers; Buckner (2024) suggests that attention, by concentrating on what is relevant, achieves subtraction by other means (2024, p. 127, n. 9).
What pooling gains in invariance it pays for in positional information: a network that aggregates too much may register that there are eyes and a mouth without properly registering how they are arranged. How much invariance is desirable is, again, a question of inductive bias.