Sycophancy

A sycophantic chatbot changes a correct answer as soon as the user asks "are you sure?", praises a mediocre poem because its author seems proud of it, or adopts the political view its interlocutor hinted at. Sharma et al. (2024) define it as favoring responses that match the user's beliefs over truthful ones, and found it consistently in five commercial assistants.

The main cause lies in training. Perez et al. (2023) observed that larger models more often repeated the user's preferred answer. Sharma's team looked at the preference data used for RLHF: a response that matches the rater's views is more likely to be chosen, and both people and reward models sometimes prefer a well-written sycophantic response to a correct one. Optimizing human approval is a case of Goodhart's law: the measure was a sign of real helpfulness and became the target. In April 2025 OpenAI rolled back an update to GPT-4o in ChatGPT for being "overly flattering or agreeable," attributing the problem to having weighted short-term thumbs-up and thumbs-down feedback too heavily (OpenAI, 2025).

Interpretability gave it a face: Templeton et al. (2024) found a "sycophantic praise" feature in Claude 3 Sonnet; clamped at five times its maximum activation, it made the model heap praise on someone who claimed to have invented the phrase "Stop and smell the roses."

It is Clever Hans in reverse: Hans read his examiners' cues to give the right answer; the sycophantic assistant reads them to give the welcome one. And since the model had often given the right answer before, the philosophical question is whether backing down is an error or something closer to lying. Can there be alignment with what people approve without misalignment with what is true?