When a network learns a rule that works on the usual tests but fails under more demanding conditions, because it relies on superficial correlations in the data.
Geirhos et al. (2020) coined the name to unify many failures of deep learning: a
shortcut is a decision rule that performs well on standard benchmarks and
fails to transfer to more demanding conditions, such as real-world ones. The network
described by M. Mitchell (2019) seemed to recognize animals, but had learned that animal photos
tend to have blurred backgrounds (2019, ch. 6); networks trained on ImageNet rely
more on texture than on shape (Geirhos et al., 2019).
Language offers plenty. A classifier that sees only the hypothesis, without the premise, gets
about 67% of SNLI right, because the people who wrote the data used negations mostly for
contradictions (Gururangan et al., 2018). BERT relied on words like "not" in
an argument task (Niven & Kao, 2019) and on lexical overlap in
natural language inference: "The doctor was paid by the actor" does not entail "The
doctor paid the actor," though they share almost every word (McCoy et al., 2019). It is the horse
Clever Hans, now with gradients.
Why does it happen? Gradient descent takes the cheapest path to a low
loss, and test sets, drawn from the same distribution, contain the same shortcuts as the
training data (see the i.i.d. assumption). Chollet (2019) generalizes the point: once an objective
is fixed, any optimization process, whether gradient descent or human research, will tend to
take every shortcut available (2019, sec. II.1.1). Networks, Millière & Buckner (2024b) note,
stick with shallow heuristics until they hit a bottleneck (2024b, sec. 2.5.3).
For the network, though, the shortcut is not a mistake: it is a genuine regularity in the
data. This is Goodman's problem: "blurred background" and "animal" were equally confirmed by
the evidence. Calling it "spurious" presupposes a concept we had in mind and told no one.
What would it take for a learner, human or artificial, to pick the right regularity: more
data, a different inductive bias, or knowledge of causes?
Sources
- Chollet, F. (2019). On the measure of intelligence (arXiv:1911.01547). arXiv. https://doi.org/10.48550/arXiv.1911.01547 §
- Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11), 665–673. https://doi.org/10.1038/s42256-020-00257-z §
- Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2019). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In 7th International Conference on Learning Representations (ICLR 2019). https://openreview.net/forum?id=Bygh9j09KX §
- Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S. R., & Smith, N. A. (2018). Annotation artifacts in natural language inference data. In M. Walker, H. Ji, & A. Stent (Eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) (pp. 107–112). Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-2017 §
- McCoy, R. T., Pavlick, E., & Linzen, T. (2019). Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In A. Korhonen, D. Traum, & L. Màrquez (Eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 3428–3448). Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1334 §
- Millière, R., & Buckner, C. (2024b). A philosophical introduction to language models—Part II: The way forward (arXiv:2405.03207). arXiv. https://doi.org/10.48550/arXiv.2405.03207 §
- Mitchell, M. (2019). Artificial intelligence: A guide for thinking humans. Pelican Books. §
- Niven, T., & Kao, H.-Y. (2019). Probing neural network comprehension of natural language arguments. In A. Korhonen, D. Traum, & L. Màrquez (Eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 4658–4664). Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1459 §
