A network trained first on a task A and then only on a task B loses what it knew about A. McCloskey & Cohen (1989) called this catastrophic interference. Their network learned the single-digit sums with a 1 and then the sums with a 2 (1989, pp. 117, 119). After a single trial on the new ones, even by the laxest criterion, it got 57% of the old ones right; after two, 30%. It answered them as sums with a 2: 6 + 1 came out as 8 (1989, pp. 122–123). Ratcliff (1990) found the same in models of recognition memory (1990, p. 285).
Why it happens. In a classical propositional network each fact has its own structure; in a connectionist network each weight takes part in the response to many inputs, so adjusting it for the new alters the old (McCloskey & Cohen, 1989, p. 147). The delta rule corrects the error on the present example without recording what earlier examples required. French (1999) treats the problem as the extreme form of the stability–plasticity dilemma (1999, p. 128).
Remedies. If the data for all tasks are interleaved, forgetting does not occur (Kirkpatrick et al., 2017). This is how deep networks are usually trained (Kumaran et al., 2016, p. 527); pretraining shuffles its corpus instead of presenting it topic by topic. When tasks arrive in sequence, old examples can be rehearsed along with the new (French, 1999, p. 132), as the hippocampus does according to complementary learning systems theory (Kumaran et al., 2016, pp. 518–519). Elastic weight consolidation (EWC), inspired by synaptic consolidation, ties each weight to its old value with a spring, stiffer the more that weight matters for A as measured by Fisher information (Kirkpatrick et al., 2017, eq. 3).
In language models. Successive fine-tuning reopens the problem. Luo et al. (2023) chained five instruction tuning tasks on models of up to 7 billion parameters, LLaMA among them: they lost general knowledge, and the larger models lost more (2023, secs. 3.1, 5.1–5.2). Mixing in general instructions softened the loss (2023, sec. 5.4).
For discussion. For McCloskey and Cohen, interference as such is not the problem: humans suffer it too. What is anomalous is its magnitude, closer to retrograde amnesia (1989, p. 124). And its cause is the same thing that makes generalization possible (1989, p. 147); sparser representations interfere less but generalize worse (French, 1999, p. 130). A flaw of sequential training, or the price of generalizing?