In current usage, logits are the vector of scores that comes out of a network's last layer before being turned into probabilities. They are unbounded (they can be negative or huge) and do not add up to anything in particular; softmax turns them into a probability distribution. In a model such as Phi-3, the language-modeling “head” produces 32,064 numbers per position, one for each token in the vocabulary; after the input “The capital of France is”, the largest one corresponds to Paris (Alammar & Grootendorst, 2024, pp. 79–81).
Tiny example. The logits [2, 1, 0] give, after softmax, ≈ [0.665, 0.245, 0.090]. The logits [12, 11, 10] give exactly the same: adding a constant to all of them changes nothing, because only the differences matter. And the difference between two logits is the logarithm of the ratio between their probabilities: 2 − 1 = 1 = ln(0.665 / 0.245).
The name is inherited. In statistics, the logit of a probability p is the logarithm of its odds, log(p / (1 − p)); logistic regression models this quantity, which ranges from −∞ to +∞, with a linear function of the attributes (Witten et al., 2017, p. 130). For instance, logit(0.9) = ln 9 ≈ 2.2. Deep learning extended the name, somewhat loosely, to any pre-softmax score.
Logits are the last thing that happens before the decision, which is why they matter to interpretability: one way to find out whether an internal representation plays a causal role is to intervene on it and watch how the logits change. This is how, for example, the board representation in Othello-GPT was studied (Millière & Buckner, 2024b, p. 17). They are also what temperature rescales before sampling.