A manifold is a subspace that, seen up close, looks like a Euclidean space: a smooth curve in the plane is a 1-dimensional manifold; a surface in space, a 2-dimensional one. The hypothesis states that natural data (faces, voices, handwritten digits) lie near a low-dimensional manifold within the huge space in which they are encoded (Chollet, 2021, sec. 5.1.2).
Evidence. It is an empirical thesis and can be tested: Fefferman, Mitter and Narayanan (2016) gave an algorithm that decides, from samples, whether some manifold of a given dimension and bounded curvature fits the data well. Pope and colleagues (2021) estimated that ImageNet images, with 224 × 224 × 3 = 150,528 values each, have an intrinsic dimension between 26 and 43, and that the number of examples needed for learning depends on that dimension, not on that of the ambient space. It is the usual answer to the curse of dimensionality.
Consequences. If it holds, learning is largely a matter of finding coordinates for the manifold, as an autoencoder does when it compresses (see latent space); and generalizing is interpolating along it, which requires a dense sampling of the manifold (Chollet, 2021, sec. 5.1.2).
Mind and brain. DiCarlo, Zoccolan and Rust (2012) describe visual recognition as the untangling of object manifolds that nuisance variation (position, lighting) leaves "tangled" together, "like pieces of paper crumpled into a ball" (Buckner, 2024, p. 115). Buckner borrows the image to think of categories as regions of a similarity space that the network transforms until they become separable (2024, pp. 114–115) (see abstraction).
A caveat for linguists. Chollet extends the hypothesis to natural language (2021, sec. 5.1.2), but text is a sequence of discrete symbols, and a manifold is a continuous object. Strictly speaking, the thesis can hold for the representations of text (embeddings, hidden states), not for the strings themselves.