ElemedexA pocket encyclopedia of AI architectures

Glossary (TM) › Language and language models

TM064

Polysemy and word-sense disambiguation

Also: homonymy, WSD, lexical ambiguity

In Spanish: Polisemia y desambiguación

A word having several related senses; disambiguation is choosing, for each occurrence, the sense that the context calls for.

A word is polysemous when it has several related senses: paper is the thing that gets wet in the rain and the company that lays off reporters. This is distinguished from homonymy, where the meanings are unrelated: bat (the animal) and bat (for baseball). Word-sense disambiguation (WSD) is the task of choosing, for each occurrence, the appropriate sense, usually from a fixed inventory of senses.

Tiny example (Bar-Hillel, 1960): "The box was in the pen." Only knowing that a box fits in a playpen and not in a writing pen decides what pen is. Bar-Hillel used it to argue that high-quality machine translation was out of reach, because it would require knowledge of the world.

Static embeddings capture neither polysemy nor homonymy: they give each word a single vector, so bank is the same by the river and in the financial system (Millière & Buckner, 2024a, sec. 2.1). Contextual embeddings were proposed, among other things, to model polysemy (Peters et al., 2018, p. 2227). ELMo solved WSD with an elementary method: average the vectors of each sense in an annotated corpus and assign each new occurrence the nearest sense; this yielded an F1 of 69.0, competitive with systems built for the task (2018, p. 2233). In BERT and its relatives, self-attention pushes the vector of the ambiguous word toward the region the context indicates, even when the cue is far away (Buckner, 2024, p. 271).

A deeper question remains. Kilgarriff (1997) titled a paper "I don't believe in word senses": senses would be the lexicographer's abstractions over clusters of uses, not discrete entities among which the speaker chooses. Contextual vectors, which vary continuously, seem to bear him out, with an echo of Wittgenstein's family resemblances (Wittgenstein, 1953/2009, §§66–67). If so, classic WSD measures the ability to reproduce a dictionary's segmentation, not necessarily the ability to understand.

Sources

  • Bar-Hillel, Y. (1960). The present status of automatic translation of languages. In F. L. Alt (Ed.), Advances in computers (Vol. 1, pp. 91–163). Academic Press. https://doi.org/10.1016/S0065-2458(08)60607-5 §
  • Buckner, C. J. (2024). From deep learning to rational machines: What the history of philosophy can teach us about the future of artificial intelligence. Oxford University Press. https://doi.org/10.1093/oso/9780197653302.001.0001 §
  • Millière, R., & Buckner, C. (2024a). A philosophical introduction to language models—Part I: Continuity with classic debates (arXiv:2401.03910). arXiv. https://doi.org/10.48550/arXiv.2401.03910 §
  • Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 2227–2237). Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1202 §
  • Wittgenstein, L. (2009). Philosophical investigations (G. E. M. Anscombe, P. M. S. Hacker, & J. Schulte, Trans.; P. M. S. Hacker & J. Schulte, Eds.; Rev. 4th ed.). Wiley-Blackwell. (Original work published 1953) §