Loading…
Glossary term
The algorithm that calculates how each parameter in a neural network contributed to the error, so it can be updated.
After a forward pass produces a prediction and a loss, backpropagation works backwards through the network. Using the chain rule from calculus, it computes how much each weight contributed to the error.
Those gradients are then used by an optimizer to update the weights.
Autograd does backpropagation for you
import torch
w = torch.tensor(2.0, requires_grad=True)
loss = (3 * w - 12) ** 2
loss.backward() # backpropagation
print(w.grad) # d(loss)/dw = 2*(3w-12)*3 = -36Backpropagation made training deep networks practical and is built into every major deep learning framework.