Learn AI /Model Quality and Evaluation /Generalisation

Generalisation

Also called: generalization, generalise.

Detail

In five seconds

What it is

The ability of a model to give useful results on new cases, not just on the examples used to build or tune it.

In plain words

In simple terms, this means a model learned something useful that still helps on new examples, not just on the examples used to build it.

An everyday example

Someone practises cycling to one address until every turn is familiar. That does not show they can reach a new address. If they have learned to read the map, recognise street signs and adjust when a road is closed, they can handle routes they have not practised. A model generalises when it likewise applies what it learned to new, relevant cases rather than merely reproducing familiar patterns.

Where the story stops working

A library assistant can consciously explain the filing system and adapt on purpose. A model does not use human reasoning in that way. Also, library wings may follow one clear catalogue rule, while real-world data can shift in messier ways, with hidden changes in language, sensors, populations or behaviour.

Remember thisJudge a model by useful performance on relevant cases it has not seen, not by how well it handles material used to build or tune it.

Understand it

Why it matters

A system can look strong in a demo, a pilot or a benchmark and still fail in real use. Generalisation matters because most decisions rely on the future, not the past examples used to build the model. If a model does not generalise, people may trust scores that do not hold across places, time periods, languages or populations. That can waste money, miss important cases, or create unfair results. For a non-specialist, this term is a practical warning: ask not only 'What score did it get?' but also 'On what data, and does that resemble where we plan to use it?'

Good score on familiar data vs good performance on new data

Show why a model must be checked on fresh data to tell whether it will work beyond development examples.

01
Build from past examples
The model is developed using labelled examples from one dataset.
Top row spanning both columns: draw a box labelled 'training data' feeding into a box labelled 'model built and tuned'.
02
Check on familiar-style data
A score measured on data very similar to development data can look impressive, but it does not prove broader usefulness.
Left column middle: arrow from 'model built and tuned' to a box labelled 'evaluation close to development data'; below it, a green score badge.
03
Check on unseen data
Use separate examples the model did not train on to test whether performance carries over.
Right column middle: arrow from 'model built and tuned' to a box labelled 'held-out or new data'; below it, a score badge.
04
Compare the outcomes
If performance stays good on unseen data, generalisation looks better. If it falls sharply, the model may not transfer well.
Bottom row: left result box says 'looks good here'; right result box splits into two small outcomes: 'score stays similar' and 'score drops'.

A two-column comparison. The left column shows a model evaluated on the same kind of data it learned from and getting a high score. The right column shows the same model on separate, unseen data, where the score either stays strong or drops. Read top to bottom in each column, then compare the final boxes.

Hover or tap each result box to reveal a short note: 'similar score suggests better generalisation' or 'large drop suggests poor generalisation or mismatch between datasets'.

Describe this without the diagram

This comparison diagram explains that generalisation is about performance on new data, not just development data. First, training data is used to build and tune a model. Then two evaluations are shown. In the left column, the model is tested on data very similar to what it learned from, and it gets a high score. In the right column, the model is tested on held-out or new data it did not train on. If the score stays similar, that suggests better generalisation. If the score drops a lot, that suggests poor generalisation or a mismatch between datasets. The main lesson is that a strong score in familiar conditions is not enough.

With reduced motion: A static two-column comparison with the same boxes and labels, no animation. Notes that would appear on hover are shown as small text under each outcome.

A model is built from past examples, often called training data. Generalisation asks whether its performance carries over to relevant examples it has not seen before.

Testing does not create generalisation. It provides evidence about it. Developers compare predictions with known answers on a separate set of examples. Strong performance on relevant unseen data suggests better generalisation. A sharp drop suggests that patterns learned during development do not transfer well.

Accuracy and generalisation are not the same. Accuracy is a measurement on a particular dataset. Generalisation concerns performance beyond the data used to train or tune the model. A model can have high development accuracy and still perform poorly in practice.

Generalisation also differs from memorisation and overfitting. Memorisation means repeating training examples or narrow patterns. Overfitting occurs when fitting the training examples closely leads to worse performance on new cases. Generalisation is usually estimated with held-out test data or cross-validation, followed by checks under relevant operating conditions.

How “Using the map, not memorising the route” maps onto the real thing

In the story	In the AI system
Practising in one wing of the library	Training on one dataset
Working in a different wing with new shelves	Being tested on new data not used in training
Using the library system rather than shelf memory	Learning broader patterns rather than memorising examples
Still finding the right books quickly	Maintaining useful performance on unseen cases

A worked example

The officer tests whether the classifier can recognise access constraints in field updates it did not encounter during development.

Starting from: A development set of earlier updates and a separate, later batch kept out of training and tuning.

The classifier is fitted and tuned only on the development set, where its precision and recall appear strong.
The later batch remains untouched until the final evaluation, preventing information from the test material leaking into development.
On the held-out batch, recall drops because several sources describe access constraints using different shorthand and sentence structures.
The officer examines the missed cases and checks whether the gap reflects overfitting, limited data coverage or a change in reporting patterns.

Result: The lower performance on held-out updates reveals a generalisation gap. Before operational use, the classifier needs broader development data and another evaluation on fresh material, not further tuning against the same test batch.

In a United Nations workplace

A humanitarian affairs officer in a country office is assessing a classifier that sorts incoming field updates by issue before preparing a briefing note for clearance at 6pm.

Strong results on updates used during development mean little if the classifier misses new wording, abbreviations or reporting styles. It should be evaluated on genuinely held-out updates from later periods and different sources, using measures that reflect the cost of missed or incorrect flags.

Before you try thisDo not enter raw field updates containing personal, protection-related or security-sensitive information into a tool that is not approved for such material. Evaluation must use an approved environment and appropriately minimised or de-identified data. Every flag still requires human review before it affects the briefing note or any follow-up.

Where you may hear it

A vendor demo before a procurement unit records its technical assessment
An ICT governance board reviewing evidence from held-out testing
A programme review where a pilot worked in one country office but not another
An evaluation office discussing whether model results transfer across reporting periods

Avoid confusion

Often confused with

Common misconceptions

Common misconception: If a model has high accuracy, it generalises well.
Correction: Not necessarily. Accuracy is only a score on a particular dataset. A model may score highly on data that is very similar to its training data, or on data affected by leakage, and still perform poorly on genuinely new cases.
Common misconception: Generalisation means the model works equally well for every group and setting.
Correction: Good generalisation usually means performance carries over better to new data overall, but that does not guarantee equal performance across languages, regions or subgroups. Those still need separate checks.
Common misconception: A very large training dataset automatically solves generalisation.
Correction: More data often helps, but only if the data is relevant and varied enough. If the data is narrow, unrepresentative or contains repeated patterns, a model can still fail on new situations.

Quick check

Which result is the best evidence that a model generalises well?

Go deeper

The simple version

In simple terms, this means a model learned something useful that still helps on new examples, not just on the examples used to build it.

The technical version

Technically, generalisation is the performance a fitted model is expected to achieve on relevant unseen data. The difference between development performance and unseen-data performance is the generalisation gap. Generalisation is estimated with held-out test sets or cross-validation, using metrics suited to the task, such as accuracy, precision, recall, F1 score or calibration. Poor unseen performance can result from overfitting, data leakage, distribution mismatch or evaluation procedures that do not reflect intended use. The central question is whether patterns fitted from the observed sample support useful performance beyond that sample.

Reviewed external resources

Primary sourceAI Risk Management Framework (opens in a new tab)NIST
This official NIST framework helps readers place model performance questions inside broader risk management and suits managers, assurance teams and technical staff.
Intermediatelink checked 2026-07-31
EncyclopaediaWikipedia search: Generalisation (opens in a new tab)Wikipedia
This Wikipedia link is a search results page, so it helps readers find several possible articles and suits someone who wants a broad starting point.
search results, not one pagelink checked 2026-07-31
CourseClassification: Accuracy, recall, precision (opens in a new tab)Google
This course resource gives an accessible explanation of common classification measures and suits readers who want to connect generalisation to practical evaluation scores.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Generalisation (opens in a new tab)YouTube search
This YouTube link is a search results page, so it lets readers choose among different video explanations and suits people who learn better by watching and listening.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Generalisation (opens in a new tab)Reddit search
This Reddit link is a search results page, so it shows community questions and debates about the term and suits readers who want informal explanations alongside caveats.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

Judge a model by useful performance on relevant cases it has not seen, not by how well it handles material used to build or tune it.