softmax(z)ᵢ = e^zᵢ / Σⱼ e^zⱼ. It takes a vector of real numbers, the logits, and returns another of the same size whose values are positive and add up to 1: a probability distribution. It preserves order (the highest score gets the highest probability) and exaggerates differences, because the exponential grows fast (Witten et al., 2017, p. 397; Chollet, 2021, sec. 4.2).
Tiny example. softmax([2, 1, 0]): e² ≈ 7.389, e¹ ≈ 2.718, e⁰ = 1; the sum is ≈ 11.107. Result: ≈ [0.665, 0.245, 0.090]. A difference of 1 between scores became a ratio of almost 3 to 1 between probabilities (e ≈ 2.718).
The name says it is a “soft” version of picking the maximum: instead of giving everything to the winner, it shares out according to the scores. It appears in key places. At the end of a language model, it turns one score per token in the vocabulary into the probability that this token comes next. Inside attention, it normalizes the relevance scores between tokens so that they add up to 1 (Alammar & Grootendorst, 2024, p. 94). And it is the standard final layer of classifiers with many classes.
Temperature T divides the logits before softmax is applied. With T = 2, the same [2, 1, 0] give ≈ [0.506, 0.307, 0.186], a flatter distribution; with T = 0.5 they give ≈ [0.867, 0.117, 0.016], a sharper one. In the limit T → 0 one recovers picking the maximum, which is greedy decoding (Alammar & Grootendorst, 2024, pp. 80, 171).
One clarification: outputs that add up to 1 are guaranteed to be probabilities formally, not to be well calibrated. A model can assign 0.99 to a wrong answer; softmax normalizes confidence, it does not justify it.