Library
Mathematics

The Intuition Behind Backpropagation

An exploration of how neural networks learn by propagating error signals backward through layers using the chain rule.

Scene 1 of 4
The Input-Output Pipeline
xNetworky
f(x,w)=yf(x, w) = y
Imagine a neural network as a machine with thousands of knobs. We feed it an image, and it makes a guess. The goal is to adjust those knobs so the guess matches the reality.
Step-by-step solver
1

Forward Pass

Calculate the output of the network given the current weights and the input.

a(L)=σ(W(L)a(L−1)+b(L))a^{(L)} = \sigma(W^{(L)} a^{(L-1)} + b^{(L)})
2

Error Computation

Calculate the gradient of the cost function with respect to the output layer neurons.

δ(L)=∇aC⊙σ′(z(L))\delta^{(L)} = \nabla_a C \odot \sigma'(z^{(L)})
3

Backwards Propagation

Use the gradient of the current layer to calculate the gradient of the previous layer using the chain rule.

δ(l)=((W(l+1))Tδ(l+1))⊙σ′(z(l))\delta^{(l)} = ((W^{(l+1)})^T \delta^{(l+1)}) \odot \sigma'(z^{(l)})
4

Weight Update

Update the weights by moving against the calculated gradient for each layer.

W(l)←W(l)−η(δ(l)(a(l−1))T)W^{(l)} \leftarrow W^{(l)} - \eta (\delta^{(l)} (a^{(l-1)})^T)

Original question

explain to me how backpropogation works

Follow-up chat

Ask me anything about this lesson — I'll answer using what we just covered.