Where classification picks a category, regression returns a number: the price of a house, tomorrow's temperature, the time a project will take (Chollet, 2021, sec. 4.3). The simplest case is linear regression: the output is a weighted sum of the features plus a constant, and learning consists in finding the weights that best fit the data (Witten et al., 2017, p. 68). In two dimensions, it means drawing the line that best passes through a cloud of points; its usual loss function is mean squared error. If the target is a vector of numbers (the coordinates of a bounding box in an image), one speaks of vector regression (Chollet, 2021, ch. 4).
Two terminological traps. Logistic regression, despite its name, is a classification algorithm: it estimates the probability of belonging to a class (Chollet, 2021, sec. 4.3). And the word itself, Witten et al. (2017) warn, has nothing to do with the everyday sense of "returning to a previous state" (2017, p. 68). It is a historical fossil: Galton (1886) used it to describe how the height of children of very tall parents tends to "regress" towards the population mean, and the name of the phenomenon stuck to the method.
Regression is also the limiting case of neural networks: a neural network without hidden layers or activation functions is a linear model, and logistic regression is a network with no hidden units (Witten et al., 2017, p. 422). What deep networks add can be read as regression with ever more flexible functions, which explains both their power and their risk of overfitting.
A common mistake: language models do not perform regression over words. They do not predict a number that is then rounded off to a word, but a probability distribution over a discrete vocabulary.