Backpropagation (short for "backward propagation of errors") is the algorithm that makes training neural networks possible. After the network makes a prediction and the error is measured, backpropagation works backwards through the layers, using calculus (the chain rule) to compute each weight's exact share of the blame.
Why it matters
- A neural network can have millions or billions of weights.
- Backpropagation computes the gradient — the direction and amount each weight should change — for all of them efficiently in one backward pass.
- Without it, training deep networks would be computationally impossible.
The training loop
- Forward pass — run an example through the network to get a prediction.
- Loss — measure how wrong the prediction is.
- Backpropagation — compute each weight's contribution to the error.
- Gradient descent — nudge every weight to reduce the error.
This loop repeats for every batch of training data, every epoch, until the network converges.
Related Tools
Related Articles
View all articlesExplore More Machine Learning
View all termsDeep Learning
Machine learning using neural networks with multiple hidden layers, allowing models to learn increasingly abstract patterns from raw data.
Read more →Gradient Descent
The optimization algorithm that trains neural networks by repeatedly nudging weights in the direction that reduces prediction error.
Read more →Neural Network
A machine learning model made of layers of simple computing units (neurons) whose connection strengths are tuned automatically from example data.
Read more →