A multilayer perceptron (MLP) is a neural network whose artificial neurons are arranged in layers: each unit receives the outputs of every unit in the previous layer, and between input and output there is at least one "hidden" layer, so called because it has no direct connection to the environment (Witten et al., 2017, p. 263). What made it famous was its training method: backpropagation of errors.
How it works
Each unit computes a weighted sum of its inputs and applies an activation function to it. The classic perceptron used a step (0 or 1); the MLP replaces it with a smooth curve, the sigmoid, because the step is not differentiable, and without derivatives there is no gradient (Witten et al., 2017, p. 264). Today ReLU is preferred, but the requirement is the same: a nonlinearity. Without it, stacking layers is pointless, since a composition of linear transformations is another linear transformation: the "deep" network would be a linear model in disguise (Chollet, 2021, sec. 2.3.5).
Training is supervised. One picks a differentiable loss function (in the 1980s, the squared error) and nudges every weight a small step in the direction that reduces it most: gradient descent (Witten et al., 2017, pp. 265–266). The problem is knowing how much blame belongs to a weight buried in an intermediate layer, whose "correct" output nobody knows. Backpropagation solves it with the chain rule: it takes the derivative of the error with respect to the output and propagates it backwards, layer by layer, multiplying local derivatives (Witten et al., 2017, pp. 267–268; Chollet, 2021, sec. 2.4.4). Chollet offers a useful picture: two classes of data are like two sheets of paper crumpled together into a ball, and each layer uncrumples them a little until a plane can separate them (2021, sec. 2.3.6).
What changed
Compared with the Perceptron, the difference lies not in the materials but in the learning. A single perceptron cannot represent XOR, because its classes cannot be separated by a straight line. Three units can: one computes OR, another NAND, and the third the conjunction of both. In general, any formula of propositional logic can be translated into an MLP, which therefore has the same expressive power as a Decision tree (ID3/C4.5) (Witten et al., 2017, pp. 262–263): a direct echo of McCulloch and Pitts's nets. That an extra layer could solve XOR was known in Rosenblatt's day; what was missing was a rule to train it (Buckner, 2024, p. 120).
That rule has a tangled history. The differentiation technique behind it was published by Seppo Linnainmaa in 1970, Paul Werbos proposed applying it to neural networks in 1974, and it was popularized in 1986 by a group of cognitive scientists modeling how people learn (Mollick, 2026). The paper by Rumelhart et al. (1986) stressed that the hidden units come to represent important features of the task domain (1986, p. 533). In its most famous example, a network learned two isomorphic family trees, one English and one Italian; trained on 100 of the 104 possible relationships, it got the remaining 4 right, and some hidden units encoded nationality, generation or family branch, features that were nowhere explicit in the input (1986, pp. 534–535). Soon afterwards it was proved that a single hidden layer suffices to approximate any continuous function (Cybenko, 1989; Hornik, 1991), although a shallow network may need an enormous number of units for what a deeper one represents compactly (Witten et al., 2017, p. 461).
The data sheet takes these classic MLPs as its reference: the family-tree network had about 600 weights (my own count from its Figure 3); the dense network with which Chollet opens his book, for the MNIST digits, about 407,000 (2021, sec. 2.1).
Limits
Gradient descent only guarantees a local minimum, and the error surfaces of MLPs usually have many (Witten et al., 2017, p. 266), although Rumelhart and his colleagues observed that in practice this was rarely serious (1986, p. 535). With more capacity than it needs, it overfits. The architecture is chosen by trial and error, and a network with hidden layers is opaque: a black box (Witten et al., 2017, pp. 263, 269). With many sigmoid layers, the vanishing gradient problem appears as well.
By the late 1990s, SVMs had pushed it aside (Chollet, 2021, sec. 1.2.2). But it never left: its descendants are Convolutional network (LeNet) and Simple recurrent network (Elman), and inside every Transformer it is still at work, under the name of the feedforward (MLP) block block.