ElemedexA pocket encyclopedia of AI architectures
ENES

Glossary (TM) › Philosophical debates

TM196

ELIZA effect

In Spanish: Efecto ELIZA

The tendency to attribute understanding, intentions or feelings to a machine on the strength of minimal linguistic cues, as ELIZA's users did in 1966.

ELIZA was a modest program: it looked for keywords in what the user typed and handed the sentence back transformed according to a script, the most famous of which imitated a Rogerian psychotherapist. Its author, Joseph Weizenbaum, noted that some subjects had been "very hard to convince" that ELIZA was not human, and that the program showed "how easy it is to create and maintain the illusion of understanding" (Weizenbaum, 1966, p. 42). Much of the trick is supplied by the interlocutor, who makes generous assumptions about a conversational partner and rationalizes odd replies rather than give those assumptions up (1966, p. 42).

The label spread later; Hofstadter (1995) treats it as a permanent hazard for AI, and M. Mitchell & Krakauer (2023) define it as the human tendency to attribute understanding and agency to machines on the faintest hint of humanlike language or behavior. M. Mitchell (2019) illustrates it with Eugene Goostman, the bot that in 2014 fooled ten of thirty judges in a version of the Turing test by dodging hard questions: the result said more about the judges than about the machine (2019, ch. 3).

Mahowald et al. (2024) call it the "good at language → good at thought" fallacy. Because in everyday life language is a window into other minds, fluency leads us to attribute intelligence and intentionality even to rudimentary dialogue systems like ELIZA. Bender & Koller (2020) show the mechanism with their The Deep-Sea Octopus: when it fools the castaway, "it is not that O's utterances make sense, but rather, that A can make sense of them" (2020, p. 5189).

With LLMs the effect changes in scale and in status. With ELIZA the illusion was complete: there was nothing behind it. A current chatbot has real capabilities, and the problem is how much to attribute; the Sydney episode shows how far projection can go. Weizenbaum opened his paper with a maxim: "to explain is to explain away"; once a program is unmasked, "its magic crumbles away" (1966, p. 36). Is knowing the mechanism of an LLM enough to dissolve the attribution, or would that be committing the redescription fallacy (Millière & Buckner, 2024a, sec. 3)?

Sources

  • Bender, E. M., & Koller, A. (2020). Climbing towards NLU: On meaning, form, and understanding in the age of data. In D. Jurafsky, J. Chai, N. Schluter, & J. Tetreault (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 5185–5198). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.463 §
  • Hofstadter, D. R., & Fluid Analogies Research Group. (1995). Fluid concepts and creative analogies: Computer models of the fundamental mechanisms of thought. Basic Books. §
  • Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024). Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011 §
  • Millière, R., & Buckner, C. (2024a). A philosophical introduction to language models—Part I: Continuity with classic debates (arXiv:2401.03910). arXiv. https://doi.org/10.48550/arXiv.2401.03910 §
  • Mitchell, M. (2019). Artificial intelligence: A guide for thinking humans. Pelican Books. §
  • Mitchell, M., & Krakauer, D. C. (2023). The debate over understanding in AI’s large language models. Proceedings of the National Academy of Sciences, 120(13), Article e2215907120. https://doi.org/10.1073/pnas.2215907120 §
  • Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168 §