Learn AI /Model Quality and Evaluation /Variance

Variance

Detail

In five seconds

What it is

Variance describes how much a fitted model’s predictions change when the same learning procedure is trained on slightly different samples.

In plain words

Variance means the model’s results do not stay steady when you test it on slightly different data.

An everyday example

A university receives 1,000 applications and asks several clerks to sort small samples into “complete” and “incomplete”. They all use the same checklist, but each clerk gets a slightly different pile of applications. If the checklist is clear and robust, the results look similar across piles. If the checklist relies on tiny quirks, one pile may look 92% complete and another only 74%, even though the overall applicant pool is similar.

Where the story stops working

A real model does not literally follow a human checklist unless it was designed that way, and variance can appear even when the training procedure is fully automated. Also, in machine learning the instability often comes from mathematical fitting to limited data, not from people applying rules inconsistently.

Remember thisA strong average score is not enough. If refitting the same procedure on slightly different samples changes predictions substantially, the model has high variance and may not generalise reliably.

Understand it

Why it matters

A model can look strong because it scored well on one test, yet still be unreliable in practice if that score changes a lot with small data changes. Variance helps you ask a better question: not only “How high is the score?” but also “How stable is it?” This matters when choosing whether to deploy a system, compare vendors, set monitoring thresholds or trust a benchmark result. For non-specialists, variance is a warning sign against over-reading a single number. Stable performance is usually easier to defend, monitor and improve than performance that depends on a lucky split.

Low variance and high variance across repeated data splits

Be able to explain that variance is about how much model results change across repeated, slightly different samples, and why that matters for trust in an evaluation score.

01
Same task, new samples
Start with one prediction task and one modelling approach, but create several slightly different train/test splits from the same dataset.
At the top, draw one dataset box feeding into three smaller split boxes in each column. Use identical labels across columns: Split 1, Split 2, Split 3.
02
Fit the model repeatedly
Train the same kind of model on each split. Only the sampled data changes.
Under each split box, place a model icon with the same label. In the low-variance column, keep the model boundary lines visually similar; in the high-variance column, make them visibly different.
03
Compare the scores
Low variance gives similar validation scores across splits. High variance gives wide score swings, even though the task is unchanged.
Show score chips under each model. Left column example: 0.84, 0.83, 0.85. Right column example: 0.93, 0.71, 0.80. Use spacing and alignment, not colour alone, to show consistency versus spread.
04
Interpret the result
A single strong score is less convincing if repeated tests move a lot. Stable performance is usually easier to trust.
Add a bottom summary band in each column. Left: ‘stable estimate’. Right: ‘unstable estimate; possible overfitting’. Connect the score chips to the summary with thin lines.

Two side-by-side columns compare repeated evaluations of the same task. The left column shows similar scores across folds and stable decision boundaries; the right shows large score swings and shifting boundaries. Read top to bottom in each column, then compare the summary at the bottom.

Hover or tap each split to highlight its decision boundary and score, then toggle an overlay that shows the score range across splits.

Describe this without the diagram

The diagram compares two ways the same model can behave across repeated train and test splits. In the left column, three slightly different data splits produce similar fitted boundaries and nearly identical scores, showing low variance. In the right column, the same process produces noticeably different boundaries and scores that vary widely, showing high variance. The message is that variance is about stability across samples: if results swing a lot, one good score may not represent real performance on new data.

With reduced motion: A static two-column comparison replaces animation. Each column shows three fixed split-model-score stacks and a final summary label. Consistency of the left scores and spread of the right scores are shown with printed values and a simple score-range bracket.

Variance tells you how sensitive a fitted model is to changes in its training sample. If you train the same learning procedure on slightly different samples, a high-variance model may give noticeably different predictions each time. A low-variance model stays more stable.

To assess variance, repeat the same learning procedure with different training samples and compare the fitted models on the same cases or equivalent data. Fold-to-fold score changes are indirect evidence because validation folds can also differ. If the model fits accidental details specific to each training sample, its predictions may change substantially across fits.

Variance is often discussed with overfitting, but they are not identical. Overfitting means fitting training examples so closely that performance drops on new cases. High variance is one common reason for this because the fitted model is sensitive to the sample used.

Variance also differs from statistical bias and reliability. Statistical bias is systematic error caused by assumptions or structure that are too simple for the pattern. Reliability is broader consistency under comparable use. Variance specifically concerns changes caused by the training sample.

How “Different clerks, slightly different piles” maps onto the real thing

In the story	In the AI system
slightly different piles of applications	different training samples or test splits
the checklist	the model structure and learned rules
big swings in completeness rates across piles	high variance in model performance across runs

A worked example

A programme officer at Headquarters is testing a classifier that flags side-event concept notes missing required sections before they are circulated for clearance.

Starting from: A labelled archive of earlier concept notes, represented through text and layout features, is evaluated using repeated cross-validation.

The officer fits the same learning procedure several times, using slightly different training and validation samples.
The average F1 score appears acceptable, but the scores and individual flags differ markedly between folds.
An error review shows that the model relies heavily on rare template headings and formatting conventions that occur in only some samples.
The unstable features are removed, the model is simplified and the evaluation is repeated to check whether both predictions and fold-level results become more consistent.

Result: The large fold-to-fold spread is treated as evidence of high variance, rather than hidden behind the average. The revised model is considered stronger only if its performance remains reasonably stable when the training sample changes.

In a United Nations workplace

At a lightly staffed field presence where the connection drops regularly, the ICT officer must send the front office, by close of business, talking points for informals on a proposed service that predicts network outages from monitoring logs.

The officer checks whether fitting the same model on slightly different weeks of logs produces substantially different outage predictions. A respectable average score will not rescue the proposal if validation results swing widely depending on which weeks were used for training.

Before you try thisNetwork logs may reveal addresses, system topology and vulnerabilities. They must not be pasted into an external tool that is not approved for security-sensitive data, and credentials must never be included. Testing should use the approved environment, minimise exposed identifiers and restrict access to the file.

Where you may hear it

During an ICT governance board review of cross-validation results
At a vendor demo, when the procurement unit asks why performance changes between test runs
In a regional bureau validation meeting before a model pilot is cleared
While a data focal point briefs the front office on unstable evaluation metrics

Avoid confusion

Often confused with

Side-by-side

Bias vs variance (and neither is discrimination)
Why does "bias" mean three different things?

Common misconceptions

Common misconception: Variance just means the model makes many mistakes.
Correction: Not exactly. A model can have a reasonable average error rate and still have high variance if that error rate changes a lot across different samples or runs. Variance is about instability, not simply the number of errors.
Common misconception: If one test score is very high, variance is not important.
Correction: A single high score may come from a favourable split, accidental patterns in the sample or data leakage. Without checking how results change across repeated evaluations, you cannot tell whether the score is representative.
Common misconception: Variance and statistical bias are the same thing.
Correction: They describe different problems. Statistical bias refers to systematic error from assumptions that are too rigid or miss the true pattern. Variance refers to sensitivity to the particular sample used. A model can be low in one and high in the other.
Common misconception: More complex models always have lower variance because they fit the data better.
Correction: Complex models often fit the training data more closely, but that can make them more sensitive to sample-specific noise. Better fit on training data can therefore increase variance on new data rather than reduce it.

Quick check

Which result most strongly suggests high variance?

Go deeper

The simple version

Variance means the model’s results do not stay steady when you test it on slightly different data.

The technical version

In machine learning, variance is the component of error linked to sensitivity to fluctuations in the training sample. Formally, if you imagine repeatedly drawing datasets from the same underlying process and refitting the same learning algorithm, high variance means the learned function changes substantially across those draws. In practice, you often observe it indirectly through unstable validation metrics, shifting decision boundaries or large fold-to-fold differences under cross-validation. High variance is commonly associated with overfitting, because the model captures idiosyncratic features of the sample rather than reproducible signal. However, variance is not identical to overfitting, and it should be interpreted alongside bias, noise, leakage risk, class balance and the evaluation design.

Reviewed external resources

Primary sourceAI Risk Management Framework (opens in a new tab)NIST
This official framework gives a risk-management perspective on testing and reliability, and suits practitioners who need to place variance within broader AI assurance work.
Intermediatelink checked 2026-07-31
EncyclopaediaBias–variance tradeoff (opens in a new tab)Wikipedia
This link gives a broad reference overview of the bias–variance tradeoff and suits readers who want a conceptual starting point with some mathematical detail.
Intermediatelink checked 2026-07-31
CourseClassification: Accuracy, recall, precision (opens in a new tab)Google
This course page explains common classification metrics and suits readers who need context for how variance can affect the interpretation of reported scores.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Variance (opens in a new tab)YouTube search
This is a YouTube search results page for variance explanations, so it suits readers who prefer to compare several video teaching styles rather than rely on one source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Variance (opens in a new tab)Reddit search
This is a Reddit search results page for community discussion of variance, so it suits readers who want to see common confusions and informal explanations, with caution about accuracy.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

A strong average score is not enough. If refitting the same procedure on slightly different samples changes predictions substantially, the model has high variance and may not generalise reliably.