In the classic data table, each row is an example (an instance) and each column an attribute or feature (Witten et al., 2017, p. 53): age, astigmatism, humidity, the number of times "free" appears in an email. Features can be numeric or nominal, that is, categories with no order or distance, such as "sunny" or "overcast" (Witten et al., 2017, p. 54). Before deep learning, choosing and constructing good features was the most important part of the job: feature engineering (Chollet, 2021, sec. 1.2.6).
Chollet (2021) gives a clear example. To read the time from a photo of a clock one can train a network on the raw pixels, or extract by hand the angle of each hand; with those two features the problem becomes so easy that machine learning is no longer needed (2021, sec. 5.4.2). A good feature expresses the problem in a simpler way.
The great novelty of deep learning was to automate this step: the intermediate layers of a network learn their own features from raw data, all at once and guided by a single objective (Chollet, 2021, sec. 1.2.6). This is representation learning. The price is that learned features have no names: they are directions in a space of activations that must be interpreted after the fact (see superposition).
Linguists have an illustrious precedent: the distinctive features of phonology, which describe each phoneme as a bundle of binary properties such as [±voice]. There, features were explicit theoretical hypotheses; in a deep network they are by-products of training, and interpretability research tries to find out whether any of them match the categories we would use.