The delta rule adjusts each weight of a linear unit according to the error it made:
Δwᵢ = η · (d − y) · xᵢ, with y = Σⱼ wⱼxⱼ
where d is the desired output and η the learning rate. No error, no change.
Origin. Widrow & Hoff (1960) used it in ADALINE (adaptive linear), a machine "about the size of a lunch pail" that learned to classify patterns set on toggle switches (1960, p. 4). Unlike the Perceptron rule, the error is measured on the sum before the threshold (1960, p. 14). The rule then turns out to be gradient descent on the squared error, one example at a time: the derivative of that error with respect to each weight is proportional to the error times the input (1960, pp. 17–18).
The same equation in psychology. Sutton & Barto (1981) pointed out that the Widrow–Hoff rule is "essentially identical" to the Rescorla & Wagner (1972) model of classical conditioning (1981, p. 155): each input is a stimulus present or absent, each weight its associative strength, and d the unconditioned stimulus (1981, p. 156). For Rescorla and Wagner, what changes a stimulus's strength is the gap between what happens and what all the stimuli present predicted (1972, pp. 73, 76). This explains blocking: if A already predicts the shock, reinforcing the compound AX teaches X next to nothing (1972, p. 77). And mere contiguity is not enough: a tone paired with shocks produced no fear when shocks were just as likely without it (Rescorla, 1968, pp. 1–2; Rescorla & Wagner, 1972, p. 87). In their words, "organisms only learn when events violate their expectations" (1972, p. 75). Sutton and Barto add that, so understood, the perceptron models not instrumental conditioning, as is often assumed, but classical conditioning (1981, p. 156).
The return. In linear attention, the state S associates keys with values, and the
delta rule writes into it by correcting only the difference:
Sₜ = Sₜ₋₁ + βₜ·(vₜ − Sₜ₋₁kₜ)·kₜᵀ (Schlag et al., 2021, sec. 4.2). Gated DeltaNet, the linear layer
of Qwen3-Next and Qwen3.5 (SSM hybrid (Jamba, Qwen3.5)), adds forgetting and presents it as gradient descent at
inference time (S. Yang et al., 2025, sec. 3.1). Learning from prediction error, not from
mere association, thus links ADALINE, Pavlov and transformers.