A gate is a vector of numbers between 0 and 1, produced by a sigmoid from the input and the state, which is multiplied element-wise with another signal:
g = σ(W·x + U·h + b), output = g ⊙ signal
Where g is close to 0, the signal is blocked; where it is close to 1, it passes. What matters is that the decision is not programmed: g depends on the data, and its weights are learned like any others (Witten et al., 2017, p. 458).
In the LSTM. The LSTM uses three gates: input (iₜ), forget (fₜ) and output (oₜ).
The memory cell is updated as cₜ = fₜ ⊙ cₜ₋₁ + iₜ ⊙ sₜ and the hidden state as
hₜ = oₜ ⊙ tanh(cₜ) (Witten et al., 2017, pp. 458–459). This lets the network keep information for
many steps without degradation, and lets backpropagated error flow without fading: it is the
answer to the vanishing gradient (Hochreiter & Schmidhuber, 1997). The GRU simplifies the scheme with no
separate cell, performing comparably on some tasks (Witten et al., 2017, p. 460; Cho et al., 2014);
Bahdanau attention's recurrent units were also of this kind (Bahdanau et al., 2015, sec. 4.2).
Beyond recurrent networks. The idea became ubiquitous. In a mixture of experts, a "gating network" decides which experts process each token (Shazeer et al., 2017) (Mixture of experts (Mixtral)). The feedforward (MLP) block networks of many current transformers use SwiGLU, a gated activation (Alammar & Grootendorst, 2024, p. 101). And Mamba makes the parameters of its recurrence depend on the input, so as to propagate or forget information selectively (Gu & Dao, 2023).
Interpretive caution. It is convenient to describe gates as "decisions" to remember or forget. M. Mitchell (2019) stresses that their weights are learned by backpropagation just like the rest (2019, ch. 12): the architecture offers the possibility of selective forgetting; what gets forgotten, and why, is fixed by training, and it need not coincide with what a linguist would call relevant information.