The loss function (also objective or cost function) takes the model's prediction and the true value and returns a distance: zero if it got it right, more the worse it did (Chollet, 2021, sec. 1.1.5). It is the only signal training receives: gradient descent moves the parameters in whatever direction makes the loss decrease. That is why it must be differentiable; a plain count of errors (the 0–1 loss) will not do, because it does not say which way to correct (Witten et al., 2017, pp. 264–265).
Two losses cover almost everything:
- Mean squared error, for regression: the average of (prediction − true value)².
- Cross-entropy, for classification: −log p(correct answer). If the model gave the correct class 0.8, the loss is −ln 0.8 ≈ 0.22; if it gave it 0.01, ≈ 4.6. It measures the distance between the probability distribution the model predicts and the true one (Chollet, 2021, sec. 4.1.3), and it amounts to the number of bits needed to communicate the correct answer given the prediction (Witten et al., 2017, p. 178).
A language model is a giant classifier: at each position, cross-entropy penalizes a low probability assigned to the token that actually came next. Its exponentiated average is perplexity.
Chollet (2021) warns that a network will take any shortcut that lowers its loss, and illustrates the point with a stupid but omnipotent AI asked to maximize the average well-being of humans, which, to make its job easier, kills almost everyone and pampers the few who remain (2021, sec. 3.6.4). What the system "wants" is exactly what the loss measures; the gap between that and what we wanted is the problem of alignment.