ElemedexA pocket encyclopedia of AI architectures

Glossary (TM) › Machine learning fundamentals

TM004

Artificial neuron (unit)

Also: unit, node, perceptron

In Spanish: Neurona artificial (unidad)

The basic unit of a neural network: it multiplies each input by a weight, adds up the results plus a bias, and applies an activation function.

The unit computes y = f(w₁x₁ + w₂x₂ + … + wₙxₙ + b): a dot product between the input vector and the weight vector, plus a bias b, passed through an activation function f. With f = step, the unit answers yes or no; with f = sigmoid, it gives a number between 0 and 1.

An example: with weights (1, 1), bias −1.5 and a step function, the unit fires only if both inputs are 1, that is, it computes AND; with bias −0.5, it computes OR (Witten et al., 2017, p. 263). That is the idea of McCulloch and Pitts: threshold units that, once connected, compute truth functions (McCulloch & Pitts, 1943). Rosenblatt added that the weights could be learned from examples (Rosenblatt, 1958). You can try it in the perceptron lab.

The biological analogy is explicit. M. Mitchell (2019) explains it this way: a neuron sums the signals it receives from other neurons and "fires" if they exceed a threshold, with stronger connections weighing more; the perceptron imitates this with numbers (2019, ch. 1). Her homely example: deciding whether to go to a movie by weighing friends' opinions according to how much we trust their taste. One chapter later, though, she prefers to speak of units rather than simulated neurons, to the relief of any neuroscientists reading (2019, ch. 2).

The relief is justified. A biological neuron is an electrochemical system with temporal dynamics that fires in spikes; the artificial unit is a static one-line function. Chollet (2021) notes that there is no evidence the brain implements the learning mechanisms of deep networks (2021, sec. 1.1.4). The word "neuron" has done much for the field's prestige and a fair amount for its confusion: that a system is made of "neurons" says nothing, by itself, about whether it thinks. That inference belongs in the catalogue of anthropomorphism.

Sources

  • Chollet, F. (2021). Deep learning with Python (2nd ed.). Manning Publications. §
  • McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/BF02478259 §
  • Mitchell, M. (2019). Artificial intelligence: A guide for thinking humans. Pelican Books. §
  • Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408. https://doi.org/10.1037/h0042519 §
  • Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2017). Data mining: Practical machine learning tools and techniques (4th ed.). Morgan Kaufmann. https://doi.org/10.1016/C2015-0-02071-8 §