Chollet (2021) defines a representation as a different way of looking at, that is, encoding, the same data: a color image can be encoded in RGB or in HSV, and "select all red pixels" is easy in the first and hard in the second. A good representation makes the rule that solves the task simple, which is why, he argues, the central problem of machine learning is finding transformations of the data that yield useful representations (2021, sec. 1.1.3).
Witten et al. (2017) distinguish three approaches: classical learning, which predicts from features specified by the user; representation learning, which transforms those features into an intermediate representation before predicting; and deep learning, which chains many such transformations (2017, p. 418). Principal component analysis is an example without networks; the hidden layer of a multilayer perceptron, adapted jointly with the output layer, is one with networks (2017, pp. 418–419). The emblematic case in language is embeddings: word2vec learns one vector per word as a by-product of predicting which words occur together (Alammar & Grootendorst, 2024, p. 8), and BERT produces vectors that depend on context. What is learned is often reused for other tasks (transfer learning).
A terminological warning for philosophers. "Representation" here means an encoding useful for a task, not a state with semantic content and correctness conditions. Whether a network's vectors represent anything in the philosophy-of-mind sense, whether they have intentionality or merely covary with properties of the world, is a question the technical term does not answer; probing and mechanistic interpretability try to narrow it down empirically. Buckner (2024) proposes an empiricist reading of these transformations: he recasts Hume's copy principle as a transformation principle, according to which simple concepts derive from systematic, domain-general transformations of sensory impressions (2024, p. 19) (see abstraction).