Learn AI /Data and Infrastructure /Scaling laws
Scaling laws
- Deeper
- Data and Infrastructure
- Policy officers
- Technical readers
Also called: scaling hypothesis.
Back to the milestone you came from
In five seconds
What it is
Scaling laws describe the regular pattern that model performance often improves as data, parameters and compute increase, though gains usually slow and practical limits remain.
In plain words
When teams make a model larger, give it more training data, or spend more processing power on training, its results often improve in a somewhat predictable way. Scaling laws are the observed patterns that help estimate how much improvement to expect and when the gains may start to level off.
An everyday example
Leila is improving a route-planning method. She can add more recorded journeys, use a more detailed map and allow more calculation time. She tests each version on the same set of trips. At first, each increase sharply reduces wrong turns. Later, adding the same amount of effort still helps, but by less. By plotting the pattern, she can estimate whether more journey data, map detail or calculation time is likely to be worth it.
A person can consciously change study strategy, rest, or ask a teacher for help. A model does not improve through human-style understanding or intention. Also, examination practice usually involves one learner over time, while scaling laws are measured across many trained model sizes and depend heavily on data quality, architecture and evaluation method.
Remember thisScaling laws estimate how model performance tends to change as data, parameters and compute increase. They help forecast diminishing returns, but do not guarantee performance on a particular task or population.
This term is contested
The broad idea is widely used, but details are not fully settled. Researchers disagree about how far observed scaling trends generalise across model architectures, tasks, data quality conditions and future systems.
Understand it
Understand it
Why it matters
Scaling laws matter because they help organisations estimate trade-offs before spending heavily on larger models. They can inform budgeting, infrastructure planning, procurement, energy use, timelines and risk review. For policy and management audiences, the key point is simple: bigger often helps, but not endlessly, not equally for every task, and not without costs. For technical teams, they provide a way to forecast whether extra data or compute is likely to be worthwhile.
How scaling laws are estimated
After viewing this, a reader should be able to explain that scaling laws come from comparing many training runs at different sizes and looking for a predictable performance trend.
01
Choose a scale factor
Pick different amounts of training data, parameter count and compute to test.
Display three left-side input blocks labelled Data, Parameters and Compute, each with small, medium and large markers.
02
Train several model versions
Run multiple training jobs so each version uses a different combination of resources.
Connect the three inputs to a middle row of 4–5 model boxes of increasing size.
03
Measure on the same benchmark
Evaluate each trained version on the same test so the comparison is fair.
Send arrows from all model boxes to one shared evaluation box labelled Same test set.
04
Plot the trend
Place each result on a chart. Performance usually improves as scale increases, but often with slowing gains.
Show a right-side chart with points forming a curve that rises and then gradually flattens, or error falling and then flattening.
05
Use the estimate carefully
Use the trend to forecast likely benefit, cost and limits, while checking data quality and task fit.
Add a final decision box under the chart with three icons: budget, energy, and task suitability.
Show three input sliders or labelled blocks for data size, parameter count and compute feeding into repeated training runs. These runs produce test scores or error rates, which feed into a chart with a smooth improving curve that flattens. Read left to right: increase resources, train several versions, measure results, then estimate expected gains and limits.
Hovering or tapping each model box highlights its point on the chart and reveals a short note such as '10x more compute, modest gain'.
Describe this without the diagram
The diagram is a left-to-right flow with five stages. First are three inputs: data size, parameter count and compute, each shown at several levels from small to large. Second, these feed into several model training runs of increasing size. Third, all runs go to one shared evaluation step using the same test set. Fourth, the measured results are plotted on a chart where performance improves with scale but the curve gradually flattens, showing diminishing returns. Fifth, a decision box reminds the reader to use the trend for planning cost, energy use and likely benefit, not as a guarantee.
With reduced motion: A static flow diagram shows the same five stages in one view, with highlighted arrows and fixed labels instead of hover effects or animation.
Scaling laws are observed patterns about size and performance in machine learning. They ask a practical question: if you make a model bigger, train it on more data, or use more compute, how much better does it usually get? Here, parameters are the adjustable numerical values inside a model, and compute means the processing work needed for training or running it.
What goes in are changes in scale: more training data, more parameters, and more compute. What happens is researchers train many model versions at different sizes and measure their error or accuracy on a test set. When they plot the results, they often find a fairly smooth trend: larger systems tend to perform better in predictable ways over a range.
What comes out is not a guarantee, but an estimate. The estimate can help teams forecast whether spending 10 times more on training might bring a large improvement, a small one, or almost none. In many cases, the gains follow a pattern of diminishing returns, meaning each extra increase in size helps less than the previous one.
This term is often confused with scalability. Scalability asks whether a system can handle more users or work in operation. Scaling laws ask how model quality changes as training resources increase. They are also not the same as a law of nature: they are empirical patterns, which means they are measured from experiments and can break when data quality, model design, or task conditions change.
How “Improving a route planner” maps onto the real thing
| In the story | In the AI system |
|---|---|
| more practice questions and worked examples | more training data |
| more study time | more compute used for training |
| larger notebook of methods | more model parameters or capacity |
| mock test scores over several weeks | measured performance across different model scales |
A worked example
An information management officer at a regional hub is assessing whether further training of an approved multilingual summarisation model is likely to improve a monitoring report workflow.
Starting from: Training logs and held-out evaluation results are available for several versions from the same model family, trained with progressively more data, parameters and compute.
- The technical unit trains the model family at several scales while keeping the training method and evaluation set as consistent as possible.
- Each version is assessed on the same held-out material using model loss and task-specific summary quality measures.
- The observed results are plotted against data, parameter count and compute, often on logarithmic axes, and a curve is fitted to the pattern.
- The curve shows whether performance is improving regularly, whether gains are slowing, and whether adding data appears more useful than adding model capacity or compute.
- The officer uses the fitted relationship to compare expected gains, costs and uncertainty before recommending any further training.
Result: The fitted trend supports a bounded forecast rather than a promise. It may show that additional scale is worthwhile, or that the likely improvement is too small to justify the added infrastructure, cost and energy use.
In a United Nations workplace
An ICT procurement focal point at a shared services centre is finalising a note for the file before the technical evaluation closes. A bidder claims that increasing model size and compute will substantially improve multilingual classification of staff service requests.
Scaling laws help the focal point test whether the claim is supported by results from several model scales, rather than by one impressive demonstration. The fitted trend can indicate likely gains and diminishing returns, but it does not establish that the proposed system will perform equally well across languages or request types.
Before you try thisBid submissions, pricing, evaluation records and staff service requests must not be entered into an external tool that has not been approved for such material. Scaling evidence also does not replace security, privacy, bias and task-specific testing under the office's established procurement and data protection procedures.
Where you may hear it
- An ICT governance board reviewing a proposal to train a larger model
- A procurement evaluation meeting testing a bidder's performance claims
- A regional bureau budget review on compute and storage requirements
- An inter-agency working group comparing multilingual model options
Avoid confusion
Avoid confusion
Often confused with
Scalability
The ability of a system to handle more users, data or tasks without a serious drop in speed, quality, reliability or cost control.
Compute
Compute is the processing capacity used to train and run AI systems, with different needs before deployment and during everyday use.
Parameter
A parameter is a learned numerical setting inside a model that helps shape which token it is likely to produce next.
Common misconceptions
Common misconception: If a model follows scaling laws, making it bigger will always solve its weaknesses.
Correction: Scaling laws describe average performance trends over a tested range. They do not guarantee that a bigger model will fix poor data quality, unsafe outputs, weak evaluation design, or a mismatch between the model and the task.
Common misconception: Scaling laws mean there is one magic size where intelligence appears suddenly.
Correction: In many studies, the trend is gradual rather than magical: performance changes smoothly as resources increase. Some abilities may seem to appear abruptly because of how tasks are measured, but that does not mean there is a universal sharp threshold.
Common misconception: Scaling laws and scalability mean the same thing.
Correction: They address different questions. Scaling laws are about how model quality changes as training resources increase. Scalability is about whether a deployed system can handle more users, traffic or workload reliably.
Common misconception: A smooth scaling curve proves the outputs are reliable or fair.
Correction: A model can improve on a benchmark while still producing biased, misleading or unsafe outputs in real use. Reliability, fairness and suitability require separate testing, governance and human oversight.
Quick check
What do scaling laws mainly help a team do?
Scaling laws are mainly used for forecasting likely performance changes as training scale increases. They do not guarantee safety or real-world reliability, and they are different from operational scalability, which is about serving workload.
Go deeper
Go deeper
The simple version
When teams make a model larger, give it more training data, or spend more processing power on training, its results often improve in a somewhat predictable way. Scaling laws are the observed patterns that help estimate how much improvement to expect and when the gains may start to level off.
The technical version
In machine learning, scaling laws are empirical relationships between model loss or benchmark performance and resource variables such as parameter count, training data size and compute budget. Researchers estimate them by training families of models across multiple scales and fitting curves, often in log-log space, to observed outcomes. These relationships can be used to forecast expected returns from increasing one or more resources and to identify regimes where a model is under-trained, data-limited or compute-limited. However, the fitted trend depends on the architecture, optimiser, data distribution, objective function and evaluation setup. It is therefore a planning tool, not a universal guarantee.
Reviewed external resources
Primary sourceAI Risk Management Framework (opens in a new tab)NIST
This NIST framework helps policy and technical readers connect model growth decisions with risk management, governance and evaluation duties in organisational practice.
Intermediatelink checked 2026-07-31
Primary sourceOECD AI Principles (opens in a new tab)OECD
These OECD principles help public-sector and management readers place scale-related choices within wider ideas of accountability, robustness and human-centred governance.
Intermediatelink checked 2026-07-31
EncyclopaediaQuantum computing scaling laws (opens in a new tab)Wikipedia
This Wikipedia page gives broad background on scaling laws in a different technical field, which may suit readers who want the general idea of how performance can change with size but should not treat it as an AI-specific source.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Scaling laws (opens in a new tab)YouTube search
This YouTube search results page helps readers who prefer video explanations compare multiple presentations of scaling laws, but it is a search page rather than one vetted lesson.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Scaling laws (opens in a new tab)Reddit search
This Reddit search results page lets readers see community questions, debates and informal explanations about scaling laws, but it is a search page and discussion quality will vary.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Scaling laws estimate how model performance tends to change as data, parameters and compute increase. They help forecast diminishing returns, but do not guarantee performance on a particular task or population.