If there are no correct answers to imitate, what is learned? Structure. The classic examples are clustering, which groups similar cases (as k-means does); dimensionality reduction, which summarizes many variables in a few; density estimation, which models how the data are distributed; and association rules, which look for any "interesting" regularity, not just those that predict a class (Witten et al., 2017, p. 45). Witten et al. (2017) invite us to imagine the iris flower data without the species column: the 150 specimens are likely to fall into natural clusters matching the three species (2017, p. 45).
The problem is evaluation. Without labels there is no objective error rate, and the success of a clustering is often judged by how useful it seems to a human user (Witten et al., 2017, p. 45). That is why unsupervised learning was long the poor relation of supervised learning, even though it is closer to how children learn. M. Mitchell (2019) quotes Yann LeCun's remark that unsupervised learning is the dark matter of AI (2019, ch. 6).
The boundary with self-supervised learning is blurry. An autoencoder, which learns to reconstruct its own input through a bottleneck, is classed as unsupervised by Witten et al. (2017, p. 445); in more recent terminology it would be called self-supervised, because the input itself provides the target. The pretraining of language models is described both ways.
A philosophical question: is the structure in the data, or does the method put it there? Every clustering presupposes a choice of features and a similarity metric, and similarity is cheap: as Goodman observed, any two things resemble one another in infinitely many respects (Buckner, 2024, p. 201).