A function f is linear if f(x + y) = f(x) + f(y) and f(c · x) = c · f(x): it respects sums and scaling. Multiplying by a matrix is linear (or affine, if a constant term is added). The trouble is that composing affine functions yields another affine function: if f(x) = 2x and g(x) = 3x + 1, then g(f(x)) = 6x + 1. That is why a neural network of a hundred layers with nothing between them would be, mathematically, a linear model in disguise (Chollet, 2021, sec. 2.3.5).
The activation function breaks that linearity. The most widely used today, ReLU(x) = max(0, x), is as simple as it looks and yet does not respect sums: ReLU(−2) + ReLU(3) = 3, but ReLU(−2 + 3) = 1.
Tiny example: XOR. Exclusive disjunction (true when exactly one input is 1) is not linearly separable: no straight line separates {(0, 1), (1, 0)} from {(0, 0), (1, 1)}. That is why a single-layer Perceptron cannot compute it, an observation Minsky and Papert made famous (1969). With one hidden layer and ReLU it is easy: h₁ = ReLU(x₁ + x₂), h₂ = ReLU(x₁ + x₂ − 1), y = h₁ − 2 · h₂. This gives 0 for (0, 0), 1 for (1, 0) and for (0, 1), and 2 − 2 = 0 for (1, 1).
Chollet offers a helpful image: the data of two classes are like two sheets of paper crumpled together into a ball, and each layer, with its nonlinearity, is a movement of the fingers that gradually uncrumples it until the sheets can be separated (Chollet, 2021, sec. 2.3.6). Nonlinearity combined with depth is what gives networks their expressive power: it allows curved decision boundaries and representations no linear transformation could reach. It is the difference between the perceptron and the Multilayer perceptron (backprop).