Learn AI /How Models Improve /Chain rule
Chain rule
- Deeper
- How Models Improve
- Technical readers
Back to the milestone you came from
In five seconds
What it is
A calculus rule for tracing how a small change in one setting affects the final error through a chain of intermediate calculations.
In plain words
If a model reaches its final error through several steps, this rule lets you trace a small effect backwards through those steps to see how an earlier setting influenced the error.
An everyday example
Imagine three connected controls between a microphone and a speaker. A small turn of the first control changes the signal entering the second; the second changes what reaches the third; the third changes the final loudness. To estimate how that first tiny turn affects the speaker, multiply the effect at each handover. The chain rule does the same for a sequence of calculations.
A recommendation pipeline often has hand-designed stages and may include discrete ranking choices, while neural models usually contain many numerical operations and parameters. In real training, the chain rule is applied to differentiable calculations, often across thousands or millions of values at once. A document moving up or down in rank can also involve non-smooth effects, so the analogy is simpler than the mathematics used in most models.
Remember thisThe chain rule traces final error back to an earlier parameter by multiplying the local sensitivity at each step in the calculation. Backpropagation applies this repeatedly across a model.
Understand it
Understand it
Why it matters
If you work with model development, procurement or oversight, this term helps you recognise how training computes gradients. A model does not improve by magic. The training procedure estimates how a small parameter change would affect the loss, then uses that gradient to adjust the parameter. The chain rule makes this calculation possible in deep models with many layers. If local derivatives vanish, grow sharply or do not exist, training can stall, become unstable or fail. Understanding this helps you ask better questions about feasibility, stability, the training process, and why some architectures are harder to optimise than others.
How the chain rule carries error backwards
Show how a final loss is connected back to an earlier parameter by multiplying local sensitivities along the computation path.
01
1. Forward calculation
Start with a parameter, use it to compute an intermediate value, then an output, then the loss.
Draw four boxes in a row labelled parameter w, hidden h, output y, loss L. Connect them with solid arrows left to right.
02
2. Local effects
Each neighbouring pair has a local sensitivity: how much the next quantity changes if the previous one changes slightly.
Above each arrow, place labels ∂h/∂w, ∂y/∂h, and ∂L/∂y in small tags.
03
3. Work backwards
Begin at the loss and move back one step at a time, reusing the local sensitivities already attached to each link.
Add a highlighted reverse arrow path from L back to w. Show a small token moving right-to-left or, in static form, numbered reverse arrows.
04
4. Combine them
The overall effect of w on L is the product of the local effects along the path: ∂L/∂w = ∂L/∂y × ∂y/∂h × ∂h/∂w.
Show the formula beneath the chain, with matching colours linking each factor to its arrow.
A left-to-right computation chain followed by a right-to-left sensitivity trace. Read the top row as the forward pass: parameter influences hidden value, hidden value influences output, output influences loss. Read the bottom row as the backward pass: each arrow carries a local derivative, and the combined effect on the parameter is built step by step from the end back to the start.
Hover or tap each link to highlight the corresponding factor in the final formula and reveal a short note saying which two quantities that derivative connects.
Describe this without the diagram
The diagram shows four quantities in a line: parameter w, hidden value h, output y, and loss L. The forward direction goes from w to h to y to L. Each link has a local derivative: how a small change in the earlier quantity affects the next one. The backward explanation starts at loss L and moves to y, then h, then w. The key statement is that the total effect of changing w on the loss equals the product of the three local derivatives: derivative of loss with respect to output, times derivative of output with respect to hidden value, times derivative of hidden value with respect to w. This illustrates that the chain rule combines local sensitivities across intermediate steps.
With reduced motion: A static version shows the same four boxes and both directions at once: solid forward arrows for computation, dashed reverse arrows for sensitivity tracing, and a colour-coded formula under the boxes matching each derivative to one link.
Many models produce an output through several steps. One setting changes an intermediate value, that intermediate value changes another, and eventually the final loss changes. The chain rule is the mathematical rule that links those small effects together.
A small effect here usually means a derivative: a number that tells you how much one quantity changes when another changes a little. The chain rule says that if A affects B, and B affects C, then the effect of A on C can be found by combining the effect of A on B with the effect of B on C. In simple cases, you multiply these local rates of change.
In model training, what goes in are the model's calculations for one example and the loss at the end. What happens is that the training procedure works backwards through the computation, step by step, using the chain rule to assign part of the final error signal to each earlier quantity and each parameter. What comes out are gradients, which are numbers used to update the parameters.
This is closely connected to backpropagation, but they are not the same. The chain rule is the mathematical principle. Backpropagation is the efficient procedure that applies that principle across many connected calculations in a model such as a neural network.
How “A row of connected volume controls” maps onto the real thing
| In the story | In the AI system |
|---|---|
| language, topic and freshness stages | intermediate computations in a model |
| tiny change in topic weighting | small change in a parameter or intermediate value |
| effect on combined score | local derivative between neighbouring steps |
| tracing impact on ranking error | using the chain rule to connect local effects to final loss |
A worked example
At the vendor demonstration, the technical lead uses a deliberately small model: h = 3w, y = h² and L = y. The training officer asks how a change in w affects the final loss when w = 2.
Starting from: w = 2
- Forward pass: h = 3 × 2 = 6, then y = 6² = 36, so L = 36.
- Find the local derivatives: ∂h/∂w = 3, ∂y/∂h = 2h = 12 at h = 6, and ∂L/∂y = 1.
- Apply the chain rule: ∂L/∂w = (∂L/∂y) × (∂y/∂h) × (∂h/∂w) = 1 × 12 × 3 = 36.
- Near w = 2, a very small increase in w therefore produces an increase in the loss of about 36 times that amount.
Result: The gradient of the loss with respect to w is 36 at w = 2. It comes from multiplying the sensitivities at each step, rather than recalculating the whole relationship as one expression.
In a United Nations workplace
In a procurement unit, a training officer is finalising the situation report on a pilot search function for supplier training materials before the regional bulletin closes. At the vendor demonstration, the technical lead uses the chain rule to trace the ranking loss backwards through each scoring step to each trainable parameter, showing how the next model update is calculated.
The explanation gives the training officer something concrete to record: an update is not assigned to a parameter by guesswork. It is calculated by multiplying local sensitivities from the final error back through the model. That distinguishes a genuine account of backpropagation from a slide saying only that the system learns.
Before you try thisThe pilot must use an approved environment and a cleared test collection. Bid content, commercial evaluations, vendor due-diligence records and personal data must not be entered into a public or otherwise unapproved tool, because training and system logs may retain or expose procurement-sensitive information.
Where you may hear it
- During the technical portion of a vendor demonstration in a procurement meeting
- In a first reading of a model-training note by an ICT governance board
- At a briefing for programme staff before a pilot is cleared
Avoid confusion
Avoid confusion
Often confused with
Common misconceptions
Common misconception: The chain rule and backpropagation are the same thing.
Correction: They are closely related but not identical. The chain rule is a rule from calculus for combining derivatives across linked computations. Backpropagation is an efficient algorithmic procedure that uses that rule repeatedly through a network to compute gradients for many parameters.
Common misconception: The chain rule only matters in very deep neural networks.
Correction: Any model built from nested differentiable functions can use it, even a small two-step computation. Deep networks make it especially important because there are many links in the chain, but the rule itself is much broader.
Common misconception: Using the chain rule tells you which parameter caused an error in a simple, human-like sense.
Correction: What it provides is a local sensitivity: how a tiny change in a parameter would change the loss at the current point. That is not the same as a full causal explanation of behaviour across all inputs or training history.
Quick check
Why is the chain rule essential during gradient-based training?
Training needs gradients with respect to parameters, but the loss usually depends on them through intermediate calculations. The chain rule connects those local derivatives to compute parameter gradients. It does not guarantee a global minimum, choose the learning rate, or use only the derivative nearest the loss.
Go deeper
Go deeper
The simple version
If a model reaches its final error through several steps, this rule lets you trace a small effect backwards through those steps to see how an earlier setting influenced the error.
The technical version
Suppose a loss L depends on an output y, which depends on an intermediate state h, which depends on a parameter w. The chain rule gives the derivative of the composite function by composing local derivatives along the path: ∂L/∂w = ∂L/∂y · ∂y/∂h · ∂h/∂w for the scalar case. In vector-valued settings, the same idea appears through Jacobians and repeated application across graph edges. Automatic differentiation and backpropagation exploit this structure to compute gradients efficiently without separately differentiating the whole expression from scratch for each parameter. In deep networks, repeated multiplication of local derivatives is also why gradient magnitudes can shrink or grow dramatically, contributing to vanishing or exploding gradients.
Reviewed external resources
EncyclopaediaChain rule (opens in a new tab)Wikipedia
This link gives a general reference explanation of the chain rule from calculus and suits readers who want a concise formal definition with standard notation.
Intermediatelink checked 2026-07-31
CourseReducing Loss (opens in a new tab)Google
This course material shows how loss is reduced during training and suits readers who want the chain rule in the practical context of model improvement.
Intermediatelink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This video gives a visual intuition for how derivatives move through neural network calculations and suits readers who learn best from geometric and animated explanations.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Chain rule (opens in a new tab)YouTube search
This search results page helps readers find multiple video explanations of the chain rule and suits people comparing teaching styles rather than looking for one fixed source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Chain rule (opens in a new tab)Reddit search
This search results page helps readers see common questions and informal explanations about the chain rule and suits readers who want community discussion, with the usual need for careful fact-checking.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
The chain rule traces final error back to an earlier parameter by multiplying the local sensitivity at each step in the calculation. Backpropagation applies this repeatedly across a model.