The perceptron is the first neural network that learns from examples: a threshold unit like the McCulloch–Pitts neuron neuron, but with numerical weights that are adjusted every time it makes a mistake. Frank Rosenblatt, a psychologist at the Cornell Aeronautical Laboratory, introduced it in 1958 as “a probabilistic model for information storage and organization in the brain” (Rosenblatt, 1958).
How it works
Each input is multiplied by a weight, the products are summed and, if the total exceeds a threshold, the unit outputs 1 (“fires”); otherwise 0 (M. Mitchell, 2019, ch. 1), or −1 in the convention of Witten et al. (2017, p. 133). If a constant input equal to 1 (the bias) is added, the threshold becomes just another weight and the rule is simply: positive sum, first class; otherwise, the second (Witten et al., 2017, p. 132). Geometrically, the weights define a hyperplane, a linear decision boundary.
The novelty is the learning rule. Start with zero (or random) weights and go through the labeled examples: if an example of the first class is misclassified, add its values to the weight vector; if it belongs to the second class, subtract them. The output for that example then moves in the right direction, because it changes by Σ aᵢ², which is always positive (Witten et al., 2017, pp. 132–133). Mitchell stresses the kinship with Skinner's behaviorism: rewarding hits and punishing errors is what we now call supervised learning (2019, ch. 1). If the data are linearly separable, the rule converges in a finite number of steps; if not, it never terminates (Witten et al., 2017, p. 133). You can try it in the perceptron lab.
What changed
Besides learned weights, the language changed. Rosenblatt regarded McCulloch and Pitts's brain models as “logical contrivances” built for particular algorithms, and held that symbolic logic and Boolean algebra were ill-suited to studying imperfect, randomly connected networks; so he formulated his model in terms of probability theory (Rosenblatt, 1958, pp. 387–388). His “photoperceptron” had a retina of sensory units, a randomly connected association layer, and response units (1958, pp. 388–390); in modern terms, a single layer of trainable weights (Buckner, 2024, p. 120).
The scale of the promises changed too. After a Navy demonstration in July 1958, the New York Times reported the expectation of a computer able to “walk, talk, see, write, reproduce itself, and be conscious of its existence” (M. Mitchell, 2019, ch. 1).
Limits
A linear boundary can separate the cases of AND, OR and NOT, but not those of XOR (Witten et al., 2017, p. 262). In 1969 Minsky and Papert proved that the kind of problems a simple perceptron solves perfectly is very limited (Minsky & Papert, 1969). That an extra layer could handle XOR was known, and Rosenblatt himself had pointed it out; what was missing was a rule for training multilayer networks (Buckner, 2024, p. 120). According to Mitchell, what weighed most was not the mathematics but the book's speculation that the multilayer extension would be “sterile”; funding dried up, and Rosenblatt died in 1971 (2019, ch. 1). Neural networks took decades to return, until backpropagation made it possible to train several layers (Chollet, 2021, sec. 1.2.2); that required replacing the step, which is not differentiable, with a sigmoid (Witten et al., 2017, p. 264). The result is the multilayer perceptron.