Broadly, a model is interpretable if a person can understand the reasons for what it produces. Data mining distinguishes patterns expressed as a "transparent box," whose structure can be examined and reasoned about, from a black box that merely predicts (Witten et al., 2017, p. 6). A small Decision tree (ID3/C4.5) is transparent: you just read its questions. For a neural network with billions of parameters, you can print out every addition and multiplication, but a list of a billion operations is not an explanation (M. Mitchell, 2019, ch. 6).
Millière & Buckner (2024b) sort the field into three families of methods (2024b, sec. 2.3): probing, which trains classifiers to read information off the activations; attribution, which assigns each part of the input an importance score (saliency maps, attention weights); and causal intervention, which modifies internal components and measures what changes in behavior. The last one underpins mechanistic interpretability, which aims to reconstruct the algorithm the model implements.
Motivations vary too: debugging, auditing high-stakes decisions (Buckner mentions the European Union's 2018 data-protection regulation and its "right to explanation"), or finding out whether a system really has the capacities its behavior suggests. Buckner also notes that other deep networks may be the most effective tools we have for interpreting a deep network (Buckner, 2024, pp. 63–64).
The word is slippery. Lipton (2018) separated a model's transparency (being able to follow how it works) from post-hoc explanations (narratives, visualizations or examples built afterward), and Millière and Buckner warn that "interpretability" is a particularly confusing term (Millière & Buckner, 2024b, sec. 2.2, n. 4). For philosophy, the underlying question is what counts as understanding a system: predicting it, being able to intervene on it, or describing it at the right level of analysis?