Backpropagation

Backpropagation computes how much each weight in a network contributed to the output error, using the chain rule to propagate gradients from the loss back through every layer. Rumelhart, Hinton & Williams (1986) popularised it as a practical way to train multi-layer networks, making deep learning feasible.

Every model in this glossary — from CNNs to Transformers — is trained with backpropagation paired with an optimizer such as Adam.