Learn AI /How Models Improve /Loss function

Loss function

Also called: cost function, objective function.

Detail

In five seconds

What it is

A loss function turns a model’s output and the correct target into a single number that shows how wrong the model was for that example or batch.

In plain words

It is the scoring rule used during training to tell the model how bad its prediction was.

An everyday example

A weather app predicts an 80 per cent chance of rain, and it rains. A scoring rule gives that forecast a small loss. If the app had predicted only a 20 per cent chance, the loss would be much larger because the forecast was confidently wrong. Each day's forecast and actual weather produce one score. By averaging those scores over many days, the app's developers can adjust the forecasting model to reduce the kinds of error defined by the scoring rule.

Where the story stops working

A librarian may choose a scoring rule by hand and inspect rankings one by one, while a trained model usually uses a mathematically defined loss and computes updates automatically over many numerical parameters. Also, many model outputs are probabilities or continuous values, not simple ranked positions.

Remember thisA loss function turns a model output and the correct target into one number, giving optimisation a precise definition of the errors to reduce.

Understand it

Why it matters

If you hear that a model was trained successfully, the next question should be: trained to reduce what? The loss function defines what counts as a costly mistake during learning, so it shapes behaviour, trade-offs and failure patterns.

How a loss function guides training

After viewing this, a reader should be able to say that the loss function converts prediction error into a number that optimisation can use to update model parameters.

01
1. Inputs and targets
A batch contains input examples and the correct targets from the training data.
Draw a stack of three document icons labelled 'questions' beside a stack of answer labels labelled 'targets'.
02
2. Model predicts
The current model turns the inputs into outputs, such as probabilities or scores.
Draw a rounded rectangle labelled 'model parameters' with arrows from the questions to three bars labelled with example scores like 0.62, 0.24, 0.14.
03
3. Loss computes mismatch
The loss function compares the outputs with the targets and produces a single number for the batch.
Draw a small calculator-style box labelled 'loss function' receiving predictions and targets, outputting a bold number such as 'loss = 1.37'.
04
4. Optimisation traces responsibility
Training computes how changing each parameter would increase or reduce the loss.
Draw a panel with small sliders or dots connected to the model box, with red and blue mini arrows indicating direction and strength of change.
05
5. Parameters update and repeat
The optimiser nudges parameters to try to lower loss on the next batch.
Draw the model box again with a small badge 'updated', then a loop arrow back toward step 2 and a smaller example 'next loss = 1.12'.

A left-to-right step diagram with five rectangular panels connected by arrows. Each panel shows one stage in training for a single batch. Use simple icons and short labels. Place a thin feedback arrow from the final panel back to the model panel to show repetition over many batches.

On hover or tap, each step reveals a one-sentence note and highlights the corresponding arrow path; a scrubber can step through the loop repeatedly to show loss decreasing across batches.

Describe this without the diagram

The diagram shows five training steps in order. First, a batch of input examples and their correct targets enters the process. Second, the model with its current parameters produces predictions. Third, a loss function compares predictions with targets and outputs one number that summarises mismatch for the batch. Fourth, optimisation uses that loss to estimate how each parameter affected it. Fifth, the parameters are updated slightly, and the process repeats on the next batch, ideally producing a lower loss. The key message is that the loss function does not fix the model by itself; it provides the number that guides parameter updates.

With reduced motion: A static five-panel diagram with arrows between panels and a dotted loop from the final panel back to the prediction panel. Two example loss values, 1.37 before the update and 1.12 after, show the intended direction of change without animation.

A model produces an output, such as a predicted class, score or number. A loss function takes that output and compares it with the target, meaning the expected answer in the training data. The result is a number: lower usually means the model matched the target better.

What goes in depends on the task. In a spam filter, the inputs to the loss function might be a predicted probability and the true label. In a forecasting task, they might be the predicted amount and the actual amount. Different loss functions measure mismatch in different ways, so the choice affects what kinds of mistakes the model is pushed to reduce.

What happens next is important. Training does not usually change the model directly from the loss alone. Instead, optimisation methods use gradients of the loss to estimate how small changes to each model setting, often called a parameter, would change the loss. Then the parameters are adjusted to try to reduce future loss.

What comes out from the loss function is not the final judgement of model quality for real-world use. It is mainly a training signal. That is why loss is often confused with an evaluation metric such as accuracy or F1 score. They are related, but not the same: a model can be trained to lower one quantity and later assessed with another.

How “Checking rain forecasts” maps onto the real thing

In the story	In the AI system
the scoring rule used by the librarian	the loss function
how far down the correct document appears	how wrong the model output is compared with the target
average penalty across many test questions during training	batch or dataset loss
adjusting the recommender after seeing the penalties	optimisation updating model parameters

A worked example

The duty officer checks the numerical example included in the technical annex for a model that flags whether a follow-up submission is likely to contain supporting evidence.

Starting from: For one submission, the model assigns 0.80 probability to the label “supporting evidence present”. The reviewed target is 1, meaning the label is correct.

Use binary cross-entropy, a loss function suited to a two-label classification task.
Apply the loss to the model output and target: −log(0.80), which is about 0.22.
If the model had assigned only 0.20 probability to the same correct label, the loss would be −log(0.20), about 1.61. The more confident error receives the larger penalty.
Average the losses across the training batch, then calculate gradients of that average with respect to the model's parameters.
Use those gradients to update the parameters in the direction expected to reduce loss on subsequent examples.

Result: The loss converts each prediction and reviewed target into a comparable scalar. Optimisation can then update the model systematically, while separate validation and human review determine whether the pilot is useful.

In a United Nations workplace

At close of business, a duty officer in an evaluation office is clearing an annex to the office's reply on an audit finding. The annex describes a pilot model that flags follow-up submissions likely to contain supporting evidence, so an evaluation officer can review them.

For each reviewed submission, the loss function compares the model's probability with the evaluation officer's label and produces one number. Averaged across a training batch, that loss gives the optimisation algorithm an objective to reduce when updating the model's parameters.

Before you try thisConfidential evaluation records, personal data, witness material and HR or procurement-sensitive information must not be entered into a tool that is not approved for them, because this could expose protected material and compromise the evaluation process. A low loss also does not establish that evidence is sufficient or remove the need for human review.

Where you may hear it

During clearance of a technical annex on model performance
At a vendor demo before a procurement unit records its technical questions
In an ICT governance board's first reading of an AI pilot proposal
When developers review a training run with the programme focal point

Avoid confusion

Often confused with

Common misconceptions

Common misconception: Lower loss always means better real-world performance.
Correction: Lower training loss only means the model fits the chosen training objective better. Real-world usefulness also depends on evaluation metrics, data quality, generalisation and whether the chosen loss matches the actual task.
Common misconception: Loss and accuracy are the same thing.
Correction: Accuracy counts correct versus incorrect predictions after a threshold or top choice, while many loss functions use richer information such as confidence. A model can keep the same accuracy while loss changes a lot.
Common misconception: There is one standard loss function for all models.
Correction: Different tasks use different losses because different outputs need different comparisons. Regression, classification and ranking often need different mathematical forms.
Common misconception: The loss function updates the model by itself.
Correction: The loss function only produces the error signal. An optimiser, often using gradients and backpropagation, uses that signal to calculate and apply parameter updates.

Quick check

Which statement best describes the role of a loss function during training?

Go deeper

The simple version

It is the scoring rule used during training to tell the model how bad its prediction was.

The technical version

It is a scalar-valued objective applied to model outputs and targets, often differentiable, whose gradients with respect to parameters are used by optimisation algorithms to update the model and minimise expected training risk.

Reviewed external resources

EncyclopaediaLoss function (opens in a new tab)Wikipedia
This link gives a broad reference overview of loss functions, including definitions and examples, and suits readers who want a general starting point.
Intermediatelink checked 2026-07-31
CourseReducing Loss (opens in a new tab)Google
This link focuses on the practical idea of reducing loss during training, and suits readers who want a structured introductory course explanation.
Intermediatelink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This link offers a visual intuition for how neural network training uses quantities like loss, and suits readers who learn best from animation and geometry.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Loss function (opens in a new tab)YouTube search
This link is a YouTube search results page, so it helps readers browse multiple video explanations at different levels rather than relying on one source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Loss function (opens in a new tab)Reddit search
This link is a Reddit search results page, so it helps readers see common questions, confusions and informal explanations from a community discussion rather than an official reference.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

A loss function turns a model output and the correct target into one number, giving optimisation a precise definition of the errors to reduce.