Deep learning

Backpropagation

Backpropagation applies the chain rule to compute gradients through a computation graph.

forward∂L∂Lxlayer 1layer 2lossErrors flow backward, giving each weight its gradient.
Backpropagation applies the chain rule to compute gradients through a computation graph. Values shown in the worked visual are illustrative unless the labels state otherwise.

How it works

It moves from the loss toward earlier operations while reusing intermediate derivatives. The result is one gradient for every trainable parameter.

A concrete example

Illustrative: if dy/dh = 3 and dh/dw = 0.4, then dy/dw = 1.2.

Do not confuse it with Gradient descent

backpropagation computes gradients; gradient descent applies updates.

Study the full lesson