AlphaFold 2

Base stats

Compare with

Parameters: 45
Data: 42
Compute: 55
Context: 30
Generality: 8
Transparency: 30
Total: 210

Approximate, ordinal scales meant for comparing species with each other; the hard numbers are in the data sheet.

Weaknesses

With fewer than about 30 sequences in the alignment, accuracy drops sharply.
It does worse on chains whose shape depends on their partners in a complex.
It predicts a static structure, like those in the PDB: neither the molecule's motion nor the folding pathway.
It does not say why a protein takes that shape: it predicts, it does not explain (see the professor's note).

Habitat

It grew up in the Protein Data Bank, the public archive where laboratories have deposited the structures they solve for decades, and in huge databases of genetic sequences. Today its predictions cover nearly every known protein, and more than two million people in 190 countries use it.

Moves

—Predict 3D structure

—Read coevolution in an alignment

TM074Invariant point attention

—Recycling

—Estimate its own confidence (pLDDT)

TM119Self-distillation

Abilities

Evoformer

48 blocks that jointly update a representation of the alignment and one of every pair of residues, so that evolution and geometry are reasoned about together.

Triangle logic

If A is near B and B is near C, that constrains the distance between A and C: pair updates are arranged in triangles so that the table can be a real structure.

Geometric invariance

Invariant point attention gives the same result if the whole protein is rotated or moved: the frame of reference does not matter.

Anatomy

AlphaFold 2 at prediction time, following Figure 1e of Jumper et al. (2021). The sequence is supplemented with a multiple sequence alignment (MSA) of related proteins and with templates; from these come two representations, one of the alignment (sequences × residues) and one of pairs (residues × residues). The trunk of 48 Evoformer blocks refines them together; the structure module, eight blocks that share weights, moves each residue as a rigid body until it yields coordinates and a per-residue confidence. The output is fed back in three times (recycling).

AlphaFold 2 is DeepMind's neural network that predicts the three-dimensional shape of a protein from its amino-acid sequence and an alignment with related sequences. It was presented in 2020 at the CASP14 competition and published in Nature in 2021 (Jumper et al., 2021).

The problem

A protein is a chain of amino acids (there are 20 kinds) that folds into a three-dimensional structure, and that shape determines what it does (The Royal Swedish Academy of Sciences, 2024a). Determining it experimentally takes months or years: in 2021 the structures of about 100,000 distinct proteins were known, against billions of sequences (Jumper et al., 2021, p. 583). Predicting structure from sequence had been open for more than fifty years, and the biennial CASP competition assesses it blind, with structures that have just been solved but not yet released (2021, p. 583).

How it works

The input is not just the sequence: the system searches genetic databases for related proteins and builds a multiple sequence alignment (MSA); it can also use similar structures already solved as templates (Jumper et al., 2021, p. 585). The alignment matters because it stores evolutionary history: correlations between positions that mutate together betray residues that touch (2021, p. 583).

The trunk is made of 48 blocks called Evoformer, which jointly update two representations: one of the alignment (sequences × residues) and one of pairs (residues × residues). The authors frame this as inference over a graph whose edges are pairs of nearby residues (2021, p. 585). There is row-wise and column-wise self-attention over the alignment, an outer product that passes information from the alignment to the pairs, and "triangle" updates: for a table of distances to be a real structure it must respect, among other things, the triangle inequality (2021, p. 586).

The structure module, eight blocks that share weights, treats each residue as a rigid body (a rotation and a translation) that starts at the origin and gradually settles. Its key piece is invariant point attention: it adds to the usual queries, keys and values of attention 3D points expressed in each residue's local frame, so that the result does not change if the whole protein is rotated or moved (2021, pp. 585, 587). The output is the coordinates of all heavy atoms plus a per-residue confidence (pLDDT), and the whole result is fed back through the network three times to be refined: "recycling" (2021, p. 585).

It was trained on Protein Data Bank (PDB) structures released up to 30 April 2018 (Jumper et al., 2021) and then, through self-distillation, on some 350,000 structures that the network itself had predicted with high confidence (2021, p. 587).

What changed

Compared with the original Transformer, the "tokens" are residues and what matters most is the relation between pairs; geometry comes in as an inductive bias. The first version of AlphaFold (CASP13, 2018), which Buckner (2024) counts among the successes of deep convolutional networks (2024, p. 119), was, according to its authors, "a completely different model" (Jumper et al., 2021, p. 584). At CASP14, AlphaFold 2's median backbone error was 0.96 Å, against 2.8 Å for the next best method; a carbon atom is about 1.4 Å wide (2021, p. 584).

In 2024 Hassabis and Jumper received half of the Nobel Prize in Chemistry "for protein structure prediction"; the other half went to David Baker, for computational protein design (The Royal Swedish Academy of Sciences, 2024a).

Forms

AlphaFold 3 (2024) replaces the Evoformer with a "Pairformer" that does much less with the alignment, and the structure module with a diffusion module, a relative of image generators. This lets it predict complexes of proteins with DNA, RNA, small molecules and ions (Abramson et al., 2024, pp. 493–494). Becoming generative brought a vice: it can invent plausible-looking structure in regions that have none (2024, p. 496). When published, it could only be used through a non-commercial web server (2024).

Limits

Accuracy drops sharply when the alignment has fewer than about 30 sequences, and the model does worse on chains whose shape depends on their partners in a complex (Jumper et al., 2021, p. 588). Like other predictors of its kind, it returns static structures like those in the PDB, not the dynamic behavior of molecules in solution (Abramson et al., 2024, p. 498). It predicts where the chain ends up, not how it gets there.

Professor Dendrett's note

The Nobel press release says that Hassabis and Jumper solved "a 50-year-old problem" (The Royal Swedish Academy of Sciences, 2024a). The 2021 paper is more cautious: it addresses "the structure prediction component" of the protein folding problem (Jumper et al., 2021, p. 583), which also asks how and why a chain reaches that shape. AlphaFold answers the first question very well and says almost nothing about the others.

What, then, does it mean to have "solved" a scientific problem? For Hempel (1965), explanation and prediction share the same logical structure: a good explanation, had it been available in time, could have served to predict. The textbook counterexample runs the other way: the barometer predicts the storm without explaining it. Buckner (2024) suggests that these networks can find patterns too complex for us, valid for prediction but perhaps inscrutable, and gives as an example protein folding, which resisted more transparent analytic models for decades (2024, p. 140). Millière & Buckner (2024b) add that talk of "alien" intelligence in science makes sense because of pragmatic goals most of us accept: to predict and perhaps to control (2024b, sec. 4.1).

There is also a question of credit: AlphaFold learned from decades of structures solved in laboratories and deposited in the PDB. Is it a theory of folding, an instrument like a microscope, or a compressed archive of crystallography? We know far more about proteins than we did in 2019; do we understand them better?

Sources

Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., Bodenstein, S. W., Evans, D. A., Hung, C.-C., O’Neill, M., Reiman, D., Tunyasuvunakool, K., Wu, Z., Žemgulytė, A., Arvaniti, E., . . . Jumper, J. M. (2024). Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature, 630(8016), 493–500. https://doi.org/10.1038/s41586-024-07487-w §

Buckner, C. J. (2024). From deep learning to rational machines: What the history of philosophy can teach us about the future of artificial intelligence. Oxford University Press. https://doi.org/10.1093/oso/9780197653302.001.0001 §

Hempel, C. G. (1965). Aspects of scientific explanation and other essays in the philosophy of science. Free Press. §

Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., . . . Hassabis, D. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873), 583–589. https://doi.org/10.1038/s41586-021-03819-2 §

Millière, R., & Buckner, C. (2024b). A philosophical introduction to language models—Part II: The way forward (arXiv:2405.03207). arXiv. https://doi.org/10.48550/arXiv.2405.03207 §

The Royal Swedish Academy of Sciences. (2024a, October 9). The Nobel Prize in Chemistry 2024 [Press release]. NobelPrize.org. https://www.nobelprize.org/prizes/chemistry/2024/press-release/ §