An overfitted model memorizes instead of generalizing. Witten et al. (2017) give the extreme case: a rule that classifies by each example's ID code has zero error on the training data and will never get a single case outside it right (2017, p. 95). The typical pattern is that, as training proceeds, the error on the training data keeps falling while the error on held-out data first falls and then rises: at first the model is underfit (it has not yet captured the relevant patterns); later it overfits (Chollet, 2021, sec. 5.1.1). One of Chollet's examples: if the word "cherimoya" appears in a single review, and that review happens to be negative, a poorly regularized model may learn that cherimoyas are a bad sign.
Deep networks aggravate the risk, because they have the capacity to memorize even randomly assigned labels (Chollet, 2021, sec. 5.1.2); Zhang and colleagues showed this in 2017 with millions of images (Buckner, 2024, pp. 137–138). The remedies are called regularization: shrinking the model, penalizing large weights (L2 or weight decay), applying dropout, or stopping training in time (early stopping) (Witten et al., 2017, p. 269; Chollet, 2021, sec. 5.4.4).
For the philosopher of science this is the old curve-fitting problem: through n points there always passes exactly one polynomial of degree n − 1, and it is almost never the best predictor. The preference for simplicity has an operational version here, the minimum description length (MDL) principle: the best theory minimizes its own size plus that of its exceptions, the smallest "cartload of dung," in the Kepler metaphor quoted by Witten et al. (2017, pp. 197–198). Forster & Sober (1994) analyzed why simplicity can improve predictive accuracy. But current practice complicates the textbook story: deep models are vastly over-parameterized and still generalize (Chollet, 2021, sec. 5.4.3), and grokking shows networks that generalize long after they have memorized.