Hans was a horse that, in the early twentieth century, appeared to solve additions, multiplications and divisions by tapping its hoof. His story left the most quoted lesson in AI evaluation: a system can give the right answer for the wrong reasons.
The story
His owner, Wilhelm von Osten, a former mathematics teacher at a Gymnasium, exhibited him to crowds in Berlin. In September 1904 a commission that included the psychologist Carl Stumpf found no tricks and no intentional signals: the horse answered even when his owner was away. Oskar Pfungst, of the Berlin Psychological Institute, then designed the decisive control. He compared the "procedure with knowledge" (the questioner knows the answer) with the "procedure without knowledge" (nobody does). With numbers written on cards that none of those present could see, Hans was right 8% of the time; when the questioner knew them, 98% (Pfungst, 1907/1911).
The explanation, in Stumpf's words, lay in the "unintentional minimal movements of the horse's questioner" (Pfungst, 1907/1911). Questioners leaned slightly forward when Hans began to tap and, when the right number was reached, made a small involuntary upward movement of the head: the signal to stop. Pfungst confirmed this in the laboratory by playing the horse himself with twenty-five people; all but two produced the same movements without noticing.
The Hans effect in AI
M. Mitchell (2019) tells how the analogy became common currency in deep learning, with examples such as a network that seemed to recognize animals but had in fact learned that animal photos tend to have blurred backgrounds (2019, ch. 6). The most literal case was documented by Lapuschkin et al. (2019): a classifier trained on the PASCAL VOC dataset recognized "horses" largely by a source tag present in about one fifth of those photos; paste the tag onto a picture of a car, and the car became a horse. The general phenomenon is known as shortcut learning, a label popularized by Geirhos et al. (2020).
In language, the canonical case is BERT. Niven & Kao (2019) showed that its strong performance on an argument-comprehension task dropped to chance once certain lexical cues were neutralized, and McCoy et al. (2019) that in natural language inference it relied on syntactic heuristics such as "if every word of the hypothesis appears in the premise, it is an entailment" (Bender & Koller, 2020, p. 5186). Their test set, HANS, includes pairs such as "The doctor was paid by the actor" / "The doctor paid the actor," where that heuristic fails (2019, p. 3429); the acronym, though the paper does not say so, is hardly an accident. A 2019 piece cited by Bender and Koller is titled, bluntly, "NLP's Clever Hans moment has arrived" (Bender & Koller, 2020, p. 5192).
The lesson
Pfungst's control is still the template: remove the suspect cue and see whether the competence survives. Its current equivalents are adversarial datasets, out-of-distribution tests (see generalization), interpretability analyses, and vigilance about data contamination. Chollet (2019) gives the general version: once an objective is fixed, developing a solution will tend to take every shortcut available, "whether this process is gradient descent or human-driven research" (2019, sec. II.1.1). But the story has a second, less quoted moral: Hans was not stupid. He was an extraordinary reader of human behavior. He had a real skill, just not the one he was credited with.