Chollet (2021) makes clear from the start that "deep" does not refer to any deep understanding, but to the number of successive layers of representation; other possible names would have been "layered" or "hierarchical representations learning" (2021, sec. 1.1.4). Buckner (2024) gives the order of magnitude: the networks of earlier waves mostly had fewer than four layers; deep ones have at least five, and sometimes hundreds or thousands (2024, p. 50). He adds that "deep" has a dual meaning: the number of layers, and the richness of the hierarchical patterns those layers come to acquire (2024, p. 51).
What changed was not just scale. Previously, a human designed the features and the algorithm learned one or two transformations on top of them ("shallow learning"); deep learning learns all the layers jointly, guided by a single error signal, and thereby automates feature engineering (Chollet, 2021, sec. 1.2.6). Witten et al. (2017) place it as a form of representation learning with many transformation steps (2017, p. 418).
The turning point is usually dated to 2012, when AlexNet won the ImageNet competition with a convolutional network trained on gaming GPUs: the winner's top-5 accuracy jumped from 74.3% the year before to 83.6% (Chollet, 2021, sec. 1.2.5; Krizhevsky et al., 2012). Since then the term has covered almost everything: vision, speech, translation, transformers, and language models.
Philosophically, Buckner (2024) reads that success as the strongest empirical evidence for a moderate empiricism: domain-general faculties with broad biases would suffice to extract abstract knowledge from experience without specific innate ideas (2024, p. 129) (see empiricism and nativism). Critics reply that these networks generalize poorly outside their training distribution (Buckner, 2024, p. 70); the debate remains open.