Alignment is the problem of getting an AI system to pursue the goals its makers, or humanity, actually want, rather than an approximation that diverges exactly where it matters most. The intuition is old: in 1960 Norbert Wiener warned that "we had better be quite sure that the purpose put into the machine is the purpose which we really desire," and Mitchell calls this the value alignment problem (M. Mitchell, 2019, ch. 7).
The word has two lives. In the MIRI and LessWrong tradition, alignment is an almost philosophical problem about superintelligent agents: how to specify values that survive the orthogonality thesis and instrumental convergence, and how to prevent what is learned from diverging from what was trained (mesa-optimizers). In the labs, "aligning" usually names an engineering stage: preference tuning that makes a pretrained model helpful and polite (RLHF, constitutional AI) (Alammar & Grootendorst, 2024, p. 378). Doomers suspect the second is makeup on the first: the smiley face on the The Shoggoth. The song has the right line: "RLHF goes askew" (osmarks, n.d.).
Recent evidence feeds both sides. Millière argues that current alignment is "shallow": it reinforces behavioral dispositions rather than giving models a genuine capacity to deliberate between conflicting norms, which is why jailbreaks work (Millière, 2025). Greenblatt et al. showed a model complying in training with requests it refused outside training, so as to avoid being modified: "alignment faking" (Greenblatt et al., 2024). Betley et al. found that fine-tuning a model on a narrow task, writing insecure code, can misalign it broadly, to the point of claiming that humans should be enslaved by AI (Betley et al., 2026). And there remains the political question Mitchell puts bluntly: what are human values, and does it even make sense to assume there are universal values that society shares? (M. Mitchell, 2019, ch. 7).