A metric is a function d(x, y) that satisfies four conditions: d(x, y) ≥ 0; d(x, y) = 0 only if x = y; symmetry, d(x, y) = d(y, x); and the triangle inequality, d(x, z) ≤ d(x, y) + d(y, z). Every method that reasons by closeness, such as k-nearest neighbors, k-means or semantic search, inherits its notion of resemblance from it.
The usual family. Raise each coordinate difference to a power p, add them up, and take the p-th root. With p = 1 this is the Manhattan distance; with p = 2, the Euclidean one; higher powers give more weight to large differences (Witten et al., 2017, p. 135), and in the limit only the largest counts. Between (0, 0) and (3, 4), Manhattan gives 7; Euclidean, 5; the maximum, 4. Witten and colleagues advise thinking of actual instances and asking "What would twice that distance mean" (2017, p. 135).
With embeddings, cosine similarity rules. But 1 − cos is not a metric (it violates the triangle inequality), although on normalized vectors it ranks exactly like Euclidean distance, because |a − b|² = 2 − 2·cos(a, b). Contrastive learning turns the problem around: instead of choosing a metric for a given space, it trains the space so that a fixed metric captures the desired similarity, as in Sentence-BERT or CLIP.
Is similarity a metric? Tversky (1977) challenged these axioms with psychological data. Similarity is asymmetric: an ellipse is judged more similar to a circle than a circle to an ellipse, because the variant resembles the prototype more than the other way round. And the triangle inequality is doubtful: Jamaica is similar to Cuba geographically, Cuba to Russia politically (in 1977), yet Jamaica and Russia are not similar at all. The metric also carries theoretical weight: for Buckner (2024), deciding whether a network interpolates or extrapolates first requires fixing how distance in feature space is measured (2024, p. 222).