Backpropagation

Backpropagation is a foundational Machine learning algorithm for training Neural Networks by calculating how much each connection (weight) contributed to the error of a prediction, then adjusting those weights to improve future guesses.

Invented in the 1980s, backpropagation transformed Artificial Intelligence by making it practical to train large networks. Here's the essence: after a network makes a prediction, you measure how wrong it was. Then you work backwards through the network—layer by layer—computing the gradient (the direction and steepness of improvement) for each weight. This gradient tells you how to nudge each weight to reduce error next time.

The magic lies in the chain rule from calculus, which lets you efficiently reuse calculations as you move backward. Without backpropagation, training Deep neural networks would be prohibitively slow, and modern Artificial Intelligence as we know it wouldn't exist.

Though originally developed for shallow networks, backpropagation remains essential for training everything from image recognition systems to language models. It's not perfect—it can get stuck in local minima and requires careful tuning—but it remains the workhorse of neural network training.

Related

Gradient descent, Supervised learning, Deep learning, Calculus, Training (Machine learning)