ElemedexA pocket encyclopedia of AI architectures
ENES

Evolution tree

How each architecture descends from earlier ones. The horizontal axis is time (stretched in recent, crowded years); each line is an evolution, colored by the main type of the species it leads to. Hover over a line to see what made it evolve.

Generations

  1. I

    Cybernetics and symbols 1943–1969

    McCulloch–Pitts neuron, n-gram language model, Logic Theorist, Perceptron, Naive Bayes, ELIZA, k-nearest neighbors, k-means

  2. II

    Classical machine learning 1970–2005

    Expert system, Decision tree, Multilayer perceptron, Simple recurrent network, Support vector machine, Deep Blue, LSTM, Convolutional network, Neural probabilistic language model, Hopfield network

  3. III

    Deep learning 2006–2016

    AlexNet, word2vec, VAE, GAN, Seq2seq, Bahdanau attention, AlphaGo

  4. IV

    Transformers 2017–2019

    Transformer, ELMo, GPT-1, BERT, GPT-2, T5, Sentence-BERT, AlphaStar

  5. V

    Large language models 2020–2022

    GPT-3, Retrieval-augmented generation, Diffusion model, Vision Transformer, CLIP, Chinchilla, Gato, AlphaFold 2, Whisper

  6. VI

    Assistants, reasoning, and agents 2022–2026

    ChatGPT, LLaMA, GPT-4, LLM agent, Mamba, Mixture of experts, Reasoning model, Claude, Gemini, DeepSeek-V3, SSM hybrid, Diffusion language model

Other trees

Other trees of language models are in circulation, and they are best read with a different question in mind. The most reproduced is that of J. Yang et al. (2023), which sorts models into three branches by what they keep of the Transformer: encoder-only (the BERT family), encoder–decoder (T5) and decoder-only (GPT-1 and its offspring); each model's height marks its date, and sharing a branch signals a closer relationship (2023, fig. 1). Updated versions tend to add two more branches, state-space models with their hybrids and text diffusion.

Those branches are classes, not lineages. In that picture they all sprout from the transformer, although Mamba descends from recurrent networks and text diffusion models inherit both from image diffusion and from BERT's fill-in-the-blank. This tree draws descent: each line says what changed between an ancestor and its successor, and a species can have two parents.

Sources

  • Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Yin, B., & Hu, X. (2023). Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond (arXiv:2304.13712). arXiv. https://doi.org/10.48550/arXiv.2304.13712 §