Learn AI /Training and Model Adaptation /Quantisation

Quantisation

Also called: quantization, quantised model.

Detail

In five seconds

What it is

A method for storing and running a model with lower-precision numbers so it uses less memory and often runs faster, with some possible loss in accuracy.

In plain words

Think of it as packing a model's numbers into a more compact form. The model often becomes cheaper and faster to run, but some precision is lost, so performance should be checked.

An everyday example

A group of colleagues in Kathmandu are planning a trip across the city using a wall map in their office. The detailed map shows every side street and lane, but it is large and slow to read during a busy morning. They switch to a simplified transit map that rounds many road details into a few clear lines and landmarks, so they can plan faster and carry the map on a phone. The route is usually still good enough, but if they need the exact entrance to a building, the simplified map may miss a detail that the full map showed.

Where the story stops working

A map removes visual detail in a way people can inspect directly, but quantisation changes internal numerical representations throughout many calculations. A simplified map does not keep trying to reproduce the original map mathematically, while quantisation methods often use scales, zero-points, calibration data, or specialised kernels to preserve behaviour as closely as possible. Also, a map can be read by a person, but a quantised model still requires machine computation.

Remember thisQuantisation does not teach a model new capabilities. It stores and often runs the model with lower-precision numbers, reducing memory use and sometimes improving speed, at the risk of measurable quality loss.

Understand it

Why it matters

Quantisation matters because it changes whether a model is practical to deploy at all. A model that is too large or slow in its original form may become usable on cheaper or more available hardware after quantisation. For a manager or technical lead, this affects cost, latency, energy use, device choice, and whether a system can run locally rather than sending data elsewhere. But it also affects risk: if quality drops on rare languages, long prompts, or safety checks, a cheaper deployment may create hidden failures. The important decision is not whether quantisation is good or bad in general, but whether the speed and memory gains are worth the measured quality loss for a specific task and operating environment.

From high-precision model to quantised model

Show how the same model can be converted to lower-precision numbers, leading to smaller memory use and often faster inference, with a possible accuracy trade-off.

01
Original model
Start with a trained model checkpoint whose weights are stored in higher precision, such as 16-bit floating point.
Draw a large rectangle labelled 'Original checkpoint'. Inside, show many small numbered cells with varied decimal values and a memory badge reading 'larger'.
02
Choose target format
Select a lower-precision representation, such as 8-bit or 4-bit, depending on the hardware and acceptable quality loss.
Add a conversion panel with a dropdown-like label 'Target: 8-bit or 4-bit' and a note 'hardware support matters'.
03
Map values to fewer levels
Values are rounded or mapped into a smaller set of numerical levels, often with scale information so calculations can approximate the original range.
Show several different decimal values funnelled into fewer bins, plus a small tag labelled 'scale'. Use arrows from the original cells into grouped levels.
04
Quantised model
The result uses less memory and may run faster, but some tasks may show a small drop in quality or robustness.
Draw a smaller rectangle labelled 'Quantised model'. Add badges 'smaller memory' and 'often faster inference', plus a caution icon with 'possible quality trade-off'.

A left-to-right state change diagram. On the left, a model uses many high-precision values. In the middle, a conversion step groups values into fewer levels and stores scale information. On the right, the quantised model is smaller and faster to run, with a note that quality may change slightly. Read from left to right.

On hover or tap, each step reveals an example precision change such as FP16 to INT8 or 4-bit, and a short note on the trade-off.

Describe this without the diagram

The diagram shows a model before and after quantisation. First, an original checkpoint stores many high-precision weight values, which takes more memory. Second, a target lower-precision format such as 8-bit or 4-bit is chosen based on hardware and quality needs. Third, the original values are mapped into fewer numerical levels, often with extra scale information so later calculations can approximate the original ranges. Fourth, the resulting quantised model is smaller and often faster for inference, but it may lose a little accuracy or reliability on some tasks.

With reduced motion: A static four-panel diagram replaces animation. Panel one shows the larger high-precision model. Panel two shows the chosen target format. Panel three shows values grouped into fewer levels with scale information. Panel four shows the smaller quantised model with notes on faster inference and possible quality trade-off.

Models store many numerical values called weights. In many released models, those values are saved with high numerical precision, such as 16-bit or 32-bit formats. Quantisation changes those values, or the calculations that use them, into lower-precision formats such as 8-bit or 4-bit.

What goes in is usually an existing model checkpoint and a choice of target format. What happens is that ranges of values are compressed into fewer possible levels. For example, many slightly different numbers may be rounded so they share one smaller representation. Some methods also keep a small amount of extra information, such as a scale factor, so the model can approximately recover the original range during calculation.

What comes out is a smaller model that can often fit on cheaper hardware or serve more users on the same hardware. Inference means using a model to produce outputs, not training it. Quantisation is mainly used for inference, though there are training-related forms too. The trade-off is that lower precision can slightly reduce output quality, and in some cases can make a model unstable if done badly.

This is often confused with fine-tuning, pruning, or distillation. Fine-tuning changes model behaviour by further training. Pruning removes some parameters or connections. Distillation trains a new smaller model to imitate a larger one. Quantisation mainly changes how numbers are represented and computed, while trying to preserve the original behaviour as much as possible.

How “Planning a journey with a simplified transit map” maps onto the real thing

In the story	In the AI system
the detailed city map	a model stored with high-precision numerical values
the simplified transit map	a quantised model using fewer bits per value
rounding many street details into a few lines	representing many possible numbers with fewer discrete levels
faster planning on a phone	lower memory use and often faster inference on limited hardware

A worked example

Before a vendor demonstration to the ICT governance board, an information management officer tests whether a locally hosted model that tags paragraphs in cleared monitoring reports can run on the approved office server.

Starting from: Model weights stored in FP16, a representative set of cleared monitoring-report extracts, and a target of INT8 weight quantisation.

Run the original model on the test extracts and record its memory use, processing time and tagging accuracy as the baseline.
For each channel or group of weights, map the original values to discrete 8-bit levels using a scale and, where required, a zero-point.
Store the quantised values with their scale metadata. At runtime, the system uses these lower-precision values while carrying out the model’s calculations.
Run the same extracts through the quantised build and compare memory use, processing time and tagging errors, paying particular attention to rare categories and borderline paragraphs.

Result: If the smaller build fits the server and its measured quality loss remains within the pilot’s agreed threshold, the officer can recommend it for testing. The original checkpoint is retained because quantisation changes numerical representation, not the model’s underlying task, and some outputs may still degrade.

In a United Nations workplace

At a multilingual duty station, a legal officer is completing clearance of a situation report for the evening regional bulletin. An approved, locally hosted model checks whether key legal terms have been rendered consistently across the three language versions, but the full-precision model exceeds the available memory.

Quantising the model may allow it to run on the approved workstation rather than larger infrastructure. The legal officer still needs to compare the quantised build with the full-precision version, particularly on uncommon terminology, because small numerical changes can produce different wording or missed inconsistencies.

Before you try thisQuantisation does not anonymise information or make a model safe for sensitive material. Witness names, protection case details, security-sensitive annexes and other confidential content must not be entered into any tool not approved for that material. The officer must review every suggestion before clearance, especially across language versions.

Where you may hear it

An ICT governance board reviewing whether a model will fit approved local hardware
A procurement unit’s vendor demonstration when memory and latency claims reach the slide deck
A country office pilot review comparing full-precision and quantised builds
A technical clearance meeting before a locally hosted model enters testing

Avoid confusion

Often confused with

Common misconceptions

Common misconception: Quantisation means training the model on your own data.
Correction: Usually it does not. Standard post-training quantisation starts from an already trained checkpoint and changes number representation for storage and computation. Some advanced methods use calibration samples or quantisation-aware training, but that is different from fine-tuning the model to learn new behaviour from your data.
Common misconception: A quantised model is simply the same as a smaller model.
Correction: Not necessarily. The parameter count can stay the same. What changes is how each parameter is represented, such as 16 bits becoming 8 or 4 bits. Distillation creates a new smaller model with fewer parameters, while quantisation usually keeps the same architecture and mostly preserves behaviour through lower precision.
Common misconception: Lower bits always mean no meaningful quality loss.
Correction: Quality depends on the model, task, quantisation method, and hardware support. Many models work well at 8-bit and some work well at 4-bit, but some capabilities degrade first, especially on reasoning-heavy tasks, long contexts, or sensitive classification boundaries. The trade-off must be tested, not assumed.
Common misconception: Quantisation and pruning are the same optimisation.
Correction: They target different things. Quantisation reduces numerical precision. Pruning removes some weights, channels, or connections entirely. Both can reduce resource use, and they can be combined, but they change the model in different ways and have different failure modes.

Quick check

Which statement best describes quantisation in common model deployment practice?

Go deeper

The simple version

Think of it as packing a model's numbers into a more compact form. The model often becomes cheaper and faster to run, but some precision is lost, so performance should be checked.

The technical version

Quantisation converts model parameters, and sometimes activations, from higher-precision representations such as FP32 or FP16 into lower-precision forms such as INT8, 4-bit, or mixed formats. The usual mechanism is to map a continuous or high-resolution value range into a smaller discrete set using a scale and sometimes a zero-point. This can be done per tensor, per channel, or per group, with symmetric or asymmetric schemes. Post-training quantisation applies this after training, often using calibration data to estimate activation ranges. Quantisation-aware training simulates low-precision effects during training so the model learns to tolerate them. Runtime efficiency depends on whether the hardware and kernels actually support the chosen format; otherwise memory may shrink without full speed gains. In large language models, people often distinguish weight-only quantisation from schemes that also quantise activations and key-value cache data. Accuracy effects are uneven: some layers and outlier values are more sensitive, so practical systems may leave selected parts in higher precision.

Reviewed external resources

EncyclopaediaQuantization (opens in a new tab)Wikipedia
This Wikipedia page gives a broad background on quantisation as a computing concept and suits readers who want a general starting point before model-specific details.
Intermediatelink checked 2026-07-31
CourseLLM Course (opens in a new tab)Hugging Face
This Hugging Face course material is useful for readers who want practical, model-focused explanations and is well suited to technical learners working with modern language models.
Deeperlink checked 2026-07-31
CourseShort Courses (opens in a new tab)DeepLearning.AI
This DeepLearning.AI short course resource suits learners who prefer guided instruction and practical intuition about deploying or optimising models.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Quantisation (opens in a new tab)YouTube search
This YouTube search results page helps readers find multiple video explanations of quantisation and suits people who learn best by comparing presenters, but it is a search page rather than one vetted lesson.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Quantisation (opens in a new tab)Reddit search
This Reddit search results page lets readers browse community questions, troubleshooting and opinions about quantisation, which suits practitioners seeking real-world discussion, but it is a search page rather than a curated reference.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

Quantisation does not teach a model new capabilities. It stores and often runs the model with lower-precision numbers, reducing memory use and sometimes improving speed, at the risk of measurable quality loss.