Library
Mathematics

Understanding Backpropagation: The Geometry of Learning

An intuitive exploration of backpropagation, treating neural networks as composition of functions and learning as a descent through a high-dimensional error landscape.

Scene 1 of 4
The Neural Network as a Composite
Neural Network Flowf1f2fL
y=fL(...f2(f1(x,w1),w2)...,wL)y = f_L(...f_2(f_1(x, w_1), w_2)..., w_L)
Think of a neural network as a long chain of function compositions. Each layer is just a transformation, taking inputs and nudging them through weights.
Step-by-step solver
1

Compute Forward Pass

Calculate activations layer by layer to obtain the final prediction.

al=σ(Wlal−1+bl)a^l = \sigma(W^l a^{l-1} + b^l)
2

Compute Error at Output

Calculate the gradient of the cost function with respect to the output layer activations.

δL=∇aC⊙σ′(zL)\delta^L = \nabla_a C \odot \sigma'(z^L)
3

Propagate Error Backwards

Use the weight matrices to transport the error signal from layer L down to layer 1.

δl=((Wl+1)Tδl+1)⊙σ′(zl)\delta^l = ((W^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l)
4

Update Weights

Adjust weights by the product of the incoming activation and the local error gradient.

∂C∂Wl=δl(al−1)T\frac{\partial C}{\partial W^l} = \delta^l (a^{l-1})^T

Original question

explain to me backpropogation very in depth at uni level, clear visuals

Follow-up chat

Ask me anything about this lesson — I'll answer using what we just covered.