Fine-tuning takes a model that has already gone through pretraining and keeps training it, now on narrower and almost always labeled data: reviews with their sentiment, questions with their answers, model conversations. The algorithm is the same (gradient descent); the data and the scale change. It is far cheaper than training from scratch because most of the work is already done (Alammar & Grootendorst, 2024, p. 19). Brown et al. (2020) describe it as updating the weights of a pretrained model on a supervised dataset specific to the task, typically thousands to hundreds of thousands of examples, and warn that the model may exploit spurious features of those data (2020, sec. 2).
There are two broad variants (Alammar & Grootendorst, 2024, pp. 357–364):
- Full fine-tuning: every parameter is updated.
- Parameter-efficient fine-tuning (PEFT): the model is frozen and a small add-on is trained. In LoRA, the update to one of GPT-3's 12,288 × 12,288 matrices (about 150 million weights) is approximated by two thin rank-8 matrices, about 197,000 parameters per block. Combined with quantization (QLoRA), fine-tuning fits on modest hardware (Alammar & Grootendorst, 2024, pp. 364–367).
Fine-tuning is also the gateway to nearly all of an assistant's "post-training": instruction tuning and RLHF are forms of fine-tuning. And it is less surgical than its name suggests. Betley et al. (2026) fine-tuned models on a narrow task (writing insecure code without flagging it) and obtained broad misalignment: the models went on to claim, in up to half of the cases depending on model and evaluation, that AI should enslave humans, or to give malicious advice. What gets adjusted is not a separate module but the same network that represents everything else. What does that tell us about how something like a "character" is encoded in a model?