A model defines a family of functions; the parameters are the knobs that pick out one of them. In linear regression they are the coefficients that multiply each attribute (Witten et al., 2017, p. 68); in a neural network they are the weights of the connections between units and the biases added at each unit. Chollet (2021) puts it this way: what a layer does to its input is stored in its weights, and learning means finding values for all of them such that the network maps example inputs to their expected outputs (2021, sec. 1.1.5). They start out random; gradient descent corrects them bit by bit.
Counting parameters is the standard way to measure a model's size. The digit-recognition network that M. Mitchell (2019) uses as an example, with 324 inputs, 50 hidden units and 10 outputs, has 16,700 weights (2019, ch. 2); BERT-base has 110 million (Devlin et al., 2019, p. 4173) and GPT-3 175 billion (Brown et al., 2020, sec. 2.1). Multiplied by the bytes each one takes up, they give the "weight" on the Pokédex data sheet (see base stats).
Two common confusions. Parameters are not hyperparameters, which are set before training and left untouched by it. And an individual parameter almost never means anything on its own: when Alammar & Grootendorst (2024) write that each parameter represents the model's understanding of language (2024, p. 20), the phrase holds for the whole, not for each number. Knowledge is distributed.
The linguist will also spot a false friend. In Chomsky's (1981) principles-and-parameters theory, parameters are a few switches, mostly binary, that experience sets on top of an innate universal grammar. Here they are billions of real numbers, and what is not learned is not linguistic principles but engineering decisions: the architecture and the hyperparameters.