Learn AI /How Models Improve /Backpropagation

Backpropagation

Also called: backprop, back propagation.

Detail

In five seconds

What it is

A method that propagates derivatives backwards through a model to compute how changes to each weight would affect the loss.

In plain words

After calculating a model’s loss, this method works backwards to compute how changes to each setting would affect that loss.

An everyday example

Imagine a microphone connected through several sound controls before reaching a speaker. The final sound is too loud. A technician works backwards from the speaker, calculating how a small turn of each control would change the final volume, including the effect of controls earlier in the chain. Those measurements do not adjust anything by themselves. They tell the technician which controls to turn and by how much. Backpropagation similarly calculates sensitivities backwards through connected model operations, while the optimiser makes the actual adjustments.

Where the story stops working

A library error can often be traced by human judgement and clear events, while backpropagation uses mathematical derivatives on numerical operations. Also, in a neural network many weights interact at once, so responsibility is distributed more finely than in a simple chain of library tasks.

Remember thisBackpropagation applies the chain rule backwards through a model to compute each parameter's effect on the loss. An optimiser uses those gradients to update the parameters.

Understand it

Why it matters

If you hear that a model was “trained”, backpropagation is a large part of what that usually means in practice. Understanding it helps you ask better questions about whether a system can improve, why training takes time and compute, and why poor data or poor loss choices can push improvement in the wrong direction.

How error moves backwards to guide weight updates

After using this explainer, a reader should be able to say that backpropagation assigns credit or blame for an error to many weights by moving backwards through the network and computing gradients.

An interactive network shows a forward pass producing a prediction, then a backward pass sending error signals from output to hidden to input-side weights. Readers can change one weight, the target value, and the learning rate, then step through how gradients and updates change.

Readers can drag a weight slider, change the target output, adjust the learning rate, and step through forward pass, loss calculation, backward pass, and weight update one stage at a time.

Describe this without the diagram

Screen-reader version: A small neural network receives an input and makes a prediction. The system compares that prediction with a target and calculates a loss value. Then it moves backwards through the connections. For each weight, it reports a gradient: a number showing whether increasing that weight would raise or lower the loss, and by how much. Finally, it applies an update based on the learning rate and announces the new weight values and the changed prediction. Users can change one weight, the target, and the learning rate, then repeat the sequence.

With reduced motion: Reduced-motion version: the same network appears as four static panels in order: prediction, loss, backward gradients, and updated weights. Colour and labels change between panels without animated movement.

When a neural network produces an answer, we compare that answer with the target and measure the error using a loss function. A loss function is a rule that turns “how wrong was it?” into a number. Backpropagation uses derivatives to work out how a small change to each weight would affect that loss.

It does this by moving backwards from the output layer toward the earlier layers. At each step, it calculates a gradient, which is the slope showing how much the loss would change if one weight changed slightly. This uses the chain rule from calculus, which connects small effects across many steps.

The output of backpropagation is not the final improved model by itself. What comes out is a set of gradients for the weights and biases. An optimisation method such as gradient descent then uses those gradients to nudge the settings.

So backpropagation is different from the forward pass and different from gradient descent. The forward pass makes a prediction. Backpropagation computes loss gradients from the output toward earlier layers. Gradient descent uses those gradients to update the parameters.

How “Adjusting a chain of connected sound controls” maps onto the real thing

In the story	In the AI system
book found on the wrong shelf	model output with measurable error
tracing the mistake backwards through shelf, trolley, desk, and bin	sending error information backwards through layers
estimating how much each stage contributed	computing gradients for different weights
adjusting staff habits for future returns	updating parameters during optimisation

A worked example

A small network is being trained to flag paragraphs that need human review before inclusion in a progress report. Its output ranges from 0 to 1.

Starting from: For a paragraph labelled 1.00, meaning review is required, the network predicts 0.80.

The forward pass produces 0.80, which is 0.20 below the target.
Using squared-error loss, the loss is (1.00 - 0.80)² = 0.04.
Backpropagation applies the chain rule from the loss through the output and earlier layers. Suppose the gradient for one output weight is -0.30.
With a learning rate of 0.1, the optimiser subtracts 0.1 × (-0.30) from that weight, increasing it by 0.03.
A new forward pass might produce 0.83. The precise change depends on the other parameters and the network's activations.

Result: Backpropagation computed the gradients. The optimiser then used them to update the parameters so that this training example would tend to produce a lower loss.

In a United Nations workplace

At close of business, an associate expert in an inter-agency secretariat is checking a classifier used to route implementing-partner progress reports into headings for the quarterly report to a donor. During training, when a paragraph is assigned to the wrong heading, backpropagation carries the loss signal backwards through the model and calculates a gradient for each weight and bias.

The gradients show the optimiser which parameters to adjust, and in which direction, to reduce similar classification errors. Backpropagation performs the calculation efficiently across the model. It does not decide whether the donor report is accurate, complete or ready for clearance.

Before you try thisTrain and test only in an approved environment using data cleared for that purpose. Progress reports may contain personal, operational or partner-sensitive information. Such material must not be entered into an unapproved tool because its storage, reuse and disclosure cannot be adequately controlled.

Where you may hear it

A technical briefing before an internal model is approved for testing
A vendor demonstration to an ICT governance board
A review meeting on why model training has stalled
A discussion with an evaluation office about how a model was developed

Avoid confusion

Often confused with

Common misconceptions

Common misconception: Backpropagation and gradient descent are the same thing.
Correction: They work together but are different. Backpropagation computes gradients, while gradient descent or another optimiser uses those gradients to update parameters.
Common misconception: Backpropagation sends the original data backwards through the model.
Correction: What moves backward is error information and derivative values, not the raw input data in the ordinary sense.
Common misconception: Backpropagation guarantees the best possible model.
Correction: It only provides local slope information for the current parameters. Results also depend on the architecture, data, loss function, optimiser, learning rate, and whether training gets stuck or unstable.
Common misconception: Only the last layer matters because the error appears at the output.
Correction: Earlier layers also affect the output through chains of computation, so backpropagation distributes responsibility across many layers using the chain rule.

Quick check

What does backpropagation mainly produce during neural network training?

Go deeper

The simple version

After calculating a model’s loss, this method works backwards to compute how changes to each setting would affect that loss.

The technical version

Backpropagation applies the chain rule to a computational graph to efficiently compute partial derivatives of the loss with respect to each parameter, propagating error signals from outputs to earlier layers so an optimiser can update weights and biases.

Reviewed external resources

EncyclopaediaBackpropagation (opens in a new tab)Wikipedia
This article gives a broad reference overview with terminology and mathematical detail, and suits readers who want a more formal summary.
Intermediatelink checked 2026-07-31
CourseReducing Loss (opens in a new tab)Google
This course section focuses on reducing loss during training and suits readers who want a practical introduction to how improvement happens.
Intermediatelink checked 2026-07-31
VideoWhat is backpropagation really doing? (opens in a new tab)3Blue1Brown
This video gives a strong visual intuition for what backpropagation computes and suits readers who learn best from animated explanations.
Intermediatelink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This video explains neural networks visually more broadly, which helps readers place backpropagation within the whole training process.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Backpropagation (opens in a new tab)YouTube search
This search results page surfaces many different video explanations of backpropagation and suits readers who want to compare teaching styles rather than rely on one source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Backpropagation (opens in a new tab)Reddit search
This search results page shows community discussions and questions about backpropagation, and suits readers who want to see common confusions and informal explanations.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

Backpropagation applies the chain rule backwards through a model to compute each parameter's effect on the loss. An optimiser uses those gradients to update the parameters.