The orthogonality thesis, formulated by Nick Bostrom in 2012, holds that intelligence and final goals are independent axes: "more or less any level of intelligence could in principle be combined with more or less any final goal" (Bostrom, 2012, sec. 1.2). A highly intelligent system could have as its sole end counting the grains of sand on a beach, computing the decimals of π or maximizing paperclips, the example that gave rise to the The Paperclip Maximizer (Bostrom, 2003). "Intelligence" here means instrumental rationality: prediction, planning, finding good means to any end.
The thesis is the silent premise of much doomerism. If intelligence does not bring good values with it, we cannot expect a superintelligence to discover morality on its own: it has to be put in, and we do not know how (alignment). The song gives it the last line of a verse: "Orthogonality thesis blues" (osmarks, n.d.).
For philosophy the interest is direct. Bostrom relates it to the Humean theory of motivation (belief alone does not move us to act, a desire is needed), while noting that he does not presuppose it, and he anticipates the moral realist's objection: even if there were objective moral facts that any fully rational agent would grasp, an agent could be highly intelligent without being rational in that "thicker" sense; his example is Parfit's agent with Future-Tuesday-Indifference (Bostrom, 2012, sec. 1.2). Critics note that the thesis thereby comes close to analyticity: if "intelligence" is defined as instrumental capacity, independence from ends follows from the definition. A more recent objection is empirical: the thesis concerns the possible, not the probable. Language models are trained on human text and inherit its norms, at least on the surface; that an arbitrary agent could have any goal says nothing about what goals a system grown from our data will have. Doomers reply that this inheritance is precisely superficial (mesa-optimizer).