An existential risk (x-risk) is, in Nick Bostrom's (2002) definition, one whose adverse outcome would annihilate Earth-originating intelligent life or permanently and drastically curtail its potential. The second clause matters: extinction is not required, a permanently foreclosed future is enough. The category includes nuclear war, engineered pandemics and asteroids; for this community, AI is the leading candidate.
The concept comes from Oxford moral philosophy and effective altruism, with a longtermist argument: if the future may hold an astronomical number of lives, reducing an existential risk by even a tiny fraction is worth more than almost any other cause. From there it reached the labs. In May 2023, hundreds of researchers and executives, including those of OpenAI, Google DeepMind and Anthropic, signed a one-sentence statement from the Center for AI Safety: mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war. Yudkowsky and Soares take the argument to its harshest conclusion (Yudkowsky & Soares, 2025); the song, to its fatalistic version: "Too late now, we lit the fuse" (osmarks, n.d.).
Criticism comes in three kinds. Technical skeptics deny there is a plausible path from current systems to that scenario, and think superintelligence belongs far down the list of worries (M. Mitchell, 2019, ch. 16). Others, in Bender's line, think that talk of extinction inflates the hype: if AI can exterminate us, it is all-powerful, which is exactly what its sellers want us to believe (Santelli, 2026). And Gebru & Torres (2024) place the concept within an ideological bundle, "TESCREAL," rooted in transhumanism and eugenics. The other side replies that present and future harms do not compete, and that dismissing a small probability of irreversible harm is a miscalculation (p(doom)).