Learn AI /How Models Improve /Error and loss
Error and loss
- Starter
- How Models Improve
- Technical readers
Also called: error, loss, residual.
Back to the milestone you came from
In five seconds
What it is
Error describes a mismatch between an output and a reference. Loss is a scalar training objective for one case or an aggregate across cases.
In plain words
A model makes a guess and you compare it with the reference answer. The gap for one case is often called error, and sometimes residual if the sign matters. A training system then combines many such gaps into one number called loss, and tries to make that number smaller.
An everyday example
A technician in Kathmandu checks a water pump. The digital sensor says the pressure is 68, but a trusted manual gauge reads 72. For that reading, the gap between 68 and 72 is the error; if the technician keeps the sign, the residual is -4 because the sensor read low. After checking many pumps, the maintenance team combines all those gaps into one score that penalises big misses more strongly, so they can tune the sensor system. That combined score plays the role of loss.
A real model usually does not get corrected by a technician directly looking at one gap. During training, software uses a mathematically defined loss over many examples to adjust many internal settings at once. Also, in practice the reference answer may itself contain noise or uncertainty, while the analogy uses a trusted gauge as if it were perfect.
Remember thisA residual is a signed mismatch, so its convention matters. A loss converts predictions and references into a scalar objective, often by aggregating per-case losses, whose gradients guide training.
Understand it
Understand it
Why it matters
If you hear that a model is improving, this is usually part of what people mean: its loss is going down under a chosen measurement rule. That matters because the rule shapes the model’s behaviour. A system punished heavily for large mistakes may act differently from one that treats all mistakes more evenly. For non-specialists, this affects procurement, evaluation, and risk: a low loss does not automatically mean the model is fair, accurate in the way you care about, or suitable for real-world use. You need to ask what was measured, on what data, and whether that measurement matches the task and the harms you want to avoid.
From one mismatch to a training loss
Show how a single prediction mismatch becomes an error or residual, and how many such mismatches are combined into one loss for training.
01
1. Output and reference
Show one model output beside the reference answer for the same case.
Draw two rounded rectangles labelled 'prediction: 68' and 'reference: 72' with a brace under both saying 'same case'.
02
2. Per-example difference
Compare the two values to get the mismatch for that case.
Add an arrow to a box labelled 'difference = 68 - 72 = -4'.
03
3. Two related views
The signed value can be called a residual; the transformed size of the mismatch is often the error measure used next.
Split into two small boxes: 'residual: -4' and 'error measure: 4 or 16, depending on the rule'. Use a note: 'rule chosen by the training setup'.
04
4. Combine across examples
Training usually combines many per-example values into one loss.
Show three to five miniature example cards feeding into a final box labelled 'average loss = 9.3', then an arrow to a small gear icon labelled 'adjust model settings'.
A left-to-right sequence with four boxes. Start with a prediction and a reference answer. Compare them to get a per-example difference. Show two labelled forms: signed residual and error magnitude. Then show several examples being combined by a rule into one loss value used to update model settings.
Hover or tap each step to reveal a short note with the same numbers, and a toggle that switches the error measure from absolute difference to squared difference.
Describe this without the diagram
The diagram shows a four-step process. First, one prediction, 68, is placed next to its reference answer, 72. Second, they are compared to produce a difference of minus 4. Third, that difference is shown in two related forms: a signed residual of minus 4, and an error measure that may be 4 if using absolute difference or 16 if using squared difference. Fourth, several examples like this are combined into one average loss, shown as 9.3, and that loss is used to adjust model settings during training. The key message is that error or residual can describe one case, while loss usually combines many cases with a chosen rule.
With reduced motion: A static four-panel diagram with the same numbers and labels: prediction and reference, difference, residual versus error measure, then multiple examples combined into one loss that points to model adjustment.
A model produces an output, such as a predicted number or label. You then compare that output with a reference answer, often called the ground truth. This comparison describes how wrong the output was for that case.
Error often means a mismatch for one example. For numeric predictions, it may be signed or transformed into a non-negative value. A residual is a closely related signed difference, especially in statistics and regression. Its sign depends on whether the convention subtracts prediction from observation or observation from prediction.
A loss is a scalar value produced by a chosen loss function. It can be calculated for one example. During training, per-example losses are often aggregated into a batch loss, such as a mean. For example, the function may square each residual so that large misses count more.
So the relationship is: a prediction and reference answer are compared; an error or residual may describe their mismatch; and a loss function converts the relevant values into an objective used for training. That loss may describe one example or an aggregate across many examples.
Accuracy instead reports the share of correct labels for a classification task. Many training losses provide a more graded signal than accuracy, including when accuracy has not changed.
How “Checking a water pump sensor against a manual gauge” maps onto the real thing
| In the story | In the AI system |
|---|---|
| one sensor reading compared with the manual gauge | one prediction compared with the reference answer |
| the signed gap of -4 | a residual for one case |
| the size of the gap for that case | an individual error measure |
| the team’s combined score across many checks | the loss used to guide improvement |
A worked example
A model predicts the number of working days needed to clear three progress reports.
Starting from: Predicted clearance times [2, 5, 4] days; actual clearance times [3, 3, 4] days.
- Define each residual as predicted minus actual: 2-3=-1, 5-3=2, and 4-4=0.
- Square the residuals so their signs no longer cancel and larger misses count more: 1, 4, and 0.
- Average the squared errors: (1+4+0) / 3 = 1.67, rounded to two decimal places.
- During training, optimisation uses the gradient of this mean squared error loss to adjust the model parameters.
Result: The signed residuals are -1, 2, and 0 days. The loss for this batch is approximately 1.67 under mean squared error.
In a United Nations workplace
A chief of section in a country office is clearing a handover note for an incoming colleague by close of business. It records a pilot model that predicts how many working days implementing-partner progress reports will take to clear.
For each completed file, the predicted clearance time can be compared with the actual time. If the residual is defined as predicted minus actual, a positive value means the model overestimated. A loss function can square or otherwise transform each discrepancy and average the results across a batch. Optimisation then uses gradients of that loss to adjust the model.
Before you try thisDo not upload progress reports, personal data or partner-sensitive material to a tool that has not been approved for them. Use an approved environment and only the minimum necessary fields, such as non-identifying workflow metadata. A low loss does not make the model suitable for judging partner performance or replacing review of the file.
Where you may hear it
- A vendor demo to an ICT governance board
- A technical evaluation meeting in a procurement unit
- A data focal point briefing the front office before a pilot
- A model review with an evaluation office
Avoid confusion
Avoid confusion
Often confused with
Loss function
A loss function turns a model’s output and the correct target into a single number that shows how wrong the model was for that example or batch.
Accuracy
The share of cases a system labels correctly out of all the cases it was tested on.
Prediction
The result a model produces from input, such as a class, number, ranking or next token.
Common misconceptions
Common misconception: Error, loss, and residual are just three names for exactly the same thing.
Correction: They are related, but not identical. Residual often means a signed difference for one case, especially in regression. Error can mean a per-case mismatch, sometimes signed and sometimes not. Loss usually means the value produced by applying a chosen rule and combining one or many mismatches into the signal used for training.
Common misconception: If loss goes down, the model is definitely getting better in every practical sense.
Correction: Lower loss means the model is improving according to that specific measurement on that data. It may still perform poorly on new data, on rare cases, or on the metric people actually care about. It can also improve average performance while remaining weak or uneven for some groups or conditions.
Common misconception: Loss and accuracy are interchangeable.
Correction: Accuracy counts how often a model gets a classification right, but loss can measure how confident or how far off predictions are, even before accuracy changes. That is why training often uses loss, while reporting may also include accuracy or other task-specific metrics.
Quick check
Which statement best captures the relationship among residual, error, and loss?
Option 1 is the best summary. In practice, usage varies by field, but residual commonly keeps the sign for one example, error refers to the mismatch on one example, and loss is the aggregated value a training procedure tries to reduce.
Go deeper
Go deeper
The simple version
A model makes a guess and you compare it with the reference answer. The gap for one case is often called error, and sometimes residual if the sign matters. A training system then combines many such gaps into one number called loss, and tries to make that number smaller.
The technical version
Let a model output y-hat for a target y. A per-example discrepancy can be represented as a residual, often y minus y-hat or y-hat minus y depending on convention, so the sign matters. An error measure may then transform that discrepancy, for example with absolute value or squaring. A loss function maps predictions and targets to a scalar objective, often by computing per-example losses and aggregating them over a batch, such as a mean. Optimisation methods use gradients of that loss with respect to model parameters to update the parameters. In ordinary usage, people sometimes say 'the error' when they really mean the training loss, so context matters.
Reviewed external resources
EncyclopaediaLoss function (opens in a new tab)Wikipedia
This Wikipedia page gives a broad reference overview of loss functions and suits readers who want definitions, variants, and related terminology.
Intermediatelink checked 2026-07-31
CourseReducing Loss (opens in a new tab)Google
This Google course section gives a beginner-friendly explanation of reducing loss during training and suits readers who want a practical introduction.
Intermediatelink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This 3Blue1Brown video offers an intuitive visual explanation of neural network training and suits visual learners who want to see how reducing loss fits into the bigger process.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Error and loss (opens in a new tab)YouTube search
This YouTube link is a search results page, so it helps readers browse multiple video explanations of error and loss and pick a style that suits them.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Error and loss (opens in a new tab)Reddit search
This Reddit link is a search results page, so it suits readers who want to see community questions, informal explanations, and common points of confusion about error and loss.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
A residual is a signed mismatch, so its convention matters. A loss converts predictions and references into a scalar objective, often by aggregating per-case losses, whose gradients guide training.