The system is trained not on one task but on many, organized into episodes, and what improves is its ability to master a new task from few examples. There are two loops: an inner one that learns each task, and an outer one that adjusts the system so that the inner loop works across the whole family. In psychology, Harlow (1949) called learning sets what monkeys acquired when, after hundreds of discrimination problems, they solved new ones almost on the first try. In machine learning, MAML (Finn et al., 2017) looks for initial parameters from which a few steps of gradient descent suffice: it trains the model to be easy to fine-tune.
GPT-3 gave the term an unexpected twist: pretraining is the outer loop and in-context learning the inner one, which happens in the activations, without touching the weights (Brown et al., 2020, secs. 1, 7). Nobody designed the episodes: the corpus contains sub-tasks repeated within a single text.
Lake & Baroni (2023) made it explicit (MLC). Each episode is a pseudo-language with made-up words (dax, wif, blicket) and a randomly generated grammar that maps them to colored circles; a Transformer sees a few study examples and interprets a new instruction, with its weights frozen at test time. It produced systematic outputs at a rate (82.4%) close to the human one (80.7%) (2023, p. 117) (see systematicity). The limit, according to the authors: it generalizes to new episodes from the same distribution as the training ones, not to others (2023, p. 119).
The debate between empiricism and nativism thus moves up a floor. Whoever meta-trains does not program inductive biases: they define the family of problems and let optimization discover the useful biases (Russin et al., 2025, sec. 3.2). For Chollet (2019), meta-learning priors are not a factor to be controlled for when measuring intelligence: they are intelligence (2019, sec. II.1.3). But if they are learned from a distribution of tasks, who chose it? Is that innate, learned, or design in disguise?