In NLI a system is given a premise and a hypothesis and must decide whether the first entails the second, contradicts it, or neither (neutral) (Devlin et al., 2019, p. 4184). Large datasets such as SNLI (570,152 pairs; Bowman et al., 2015) and MultiNLI made it a central testbed for “understanding”: MNLI is part of the GLUE benchmark, and its pairs are also used to train embeddings through contrastive learning (Alammar & Grootendorst, 2024, pp. 296–297).
Tiny example (from the SNLI instructions). Premise: “Two dogs are running through a field.” “There are animals outdoors”: entailment. “Some puppies are running to catch a stick”: neutral. “The pets are sitting on a couch”: contradiction.
Why the difference matters to a logician. NLI “entailment” is not logical consequence. The founding definition is deliberately informal: T entails H if, “typically, a human reading T would infer that H is most likely true”, relying on shared background knowledge (Dagan et al., 2006). This admits defeasible, world-knowledge-laden inferences and leaves out almost all the logical consequences no annotator would think of. The labels are judgments by platform workers (data annotation), not theorems. And contradictions depend on coreference conventions: “A boat sank in the Pacific Ocean” and “A boat sank in the Atlantic Ocean” contradict each other only if they describe the same event; but if one always assumes so, “Ruth Bader Ginsburg was appointed to the US Supreme Court” would contradict “I had a sandwich for lunch today” (Bowman et al., 2015, p. 633). Even Alammar and Grootendorst's textbook example, “He is in the cinema watching Coco” versus “He is watching Frozen at home”, is a contradiction only if the pronoun and the time are the same (2024, p. 296).
Models that score well often learn shortcuts. Trained on MNLI, BERT tends to assume that a premise entails any hypothesis built from its own words, as in “The doctor was paid by the actor” → “The doctor paid the actor” (McCoy et al., 2019, p. 3429); models also fail at monotonicity inferences, especially in downward-entailing contexts (Yanaka et al., 2019). Measuring logical consequence in the strict sense takes other datasets, such as the pairs of propositional formulas of Evans et al. (2018). Against shortcuts, adversarial datasets are written by people trying to make the model fail (ANLI, Nie et al., 2020, pp. 4885–4886).
What about other languages? Almost all of this is in English. XNLI took the dataset to fifteen languages by translating it, and its authors warn that translation misses cultural differences (Conneau et al., 2018, p. 2477). InferES was written directly in European Spanish (Kovatchev & Taulé, 2022); ESPANLI, still being designed, proposes an adversarial dataset for Río de la Plata Spanish (Santelli et al., 2026). And mind the terminology: in machine learning, inference often just means running a model.