Learn AI /How Models Improve /Hyperparameter

Hyperparameter

Also called: hyperparameters, hyper-parameter.

Detail

In five seconds

What it is

A hyperparameter is a training setting chosen before or during learning that shapes how a model is fitted, rather than a value the model learns from data.

In plain words

Hyperparameters are the controls you set for training, like how big each step should be or how strongly to penalise complexity. The model then learns its own internal values under those controls.

An everyday example

A team in Vienna needs to match photographs of paper forms to the correct digital template. Before scanning the full stack, they choose the camera resolution, the brightness threshold for separating text from background, and how strict the matching score should be. Then the software processes the images and adjusts its internal matching values using examples. At the end, the team compares which setup gave the fewest mismatches on a test batch and keeps that setup for the main job.

Where the story stops working

The scanner story suggests people manually pick a few obvious settings and then the job is finished. In machine learning, there may be many interacting hyperparameters, and systems may search them automatically across many training runs. Also, some choices such as network architecture change the model structure itself, not just a simple device setting. The analogy does not show the mathematics of gradients, loss or optimisation.

Remember thisHyperparameters are the settings that steer learning; parameters are the values the model learns.

This term is contested

Some edge cases are treated differently across sources, especially whether architecture choices and certain automatically tuned settings should be counted as hyperparameters. The core idea, however, is widely agreed: they are settings that govern learning rather than ordinary learned parameters.

Understand it

Why it matters

You do not need to tune models yourself to be affected by hyperparameters. They influence whether training is stable, whether a model generalises beyond its training data, how much computing time is used, and how repeatable the results are. For managers and reviewers, this matters because a model that looks strong in one demo may perform worse with different settings, and a poor setting choice can be mistaken for a poor model design. Asking which hyperparameters were used, how they were chosen, and what validation process checked them helps you judge quality, cost, risk and fairness more sensibly.

How hyperparameters shape training

Show that hyperparameters are chosen settings that influence training runs, which then produce different learned parameters and different results.

01
1. Choose settings
Pick values such as learning rate, batch size, number of epochs or regularisation strength before a training run starts.
Draw a box labelled 'Chosen hyperparameters' with chips: learning rate 0.1, batch size 32, epochs 10, regularisation 0.01.
02
2. Train the model
The training algorithm uses those settings while repeatedly comparing predictions with known answers and computing updates.
Draw an arrow to a box labelled 'Training process' with small sublabels 'loss', 'gradient', 'update'.
03
3. Learn parameters
During training, the model's parameters change step by step. These are learned from data, unlike the hyperparameters.
Draw a box labelled 'Learned parameters' with several dials moving to new positions; note 'updated by training'.
04
4. Measure results
After training, check performance on validation data to see whether this setting choice helped or harmed.
Draw a box labelled 'Validation result' showing example outputs such as 'loss 0.42' and 'accuracy 88%'.
05
5. Tune and rerun
Compare runs, change the hyperparameters, and train again until the trade-off between quality, stability and cost is acceptable.
Draw a loop arrow from 'Validation result' back to 'Chosen hyperparameters' with note 'try new settings'.

A left-to-right step diagram with five boxes. Read from choosing settings, to running training, to learning parameters, to checking validation results, to revising the settings for the next run. Use one colour for chosen settings and another for learned values.

Click each step to highlight the difference between chosen hyperparameters and learned parameters; hovering over a setting shows one likely effect of making it larger or smaller.

Describe this without the diagram

The diagram is a five-step sequence read from left to right. Step 1 shows hyperparameters chosen before training, including learning rate, batch size, epochs and regularisation. Step 2 shows the training process using loss, gradients and updates. Step 3 shows learned parameters changing during training. Step 4 shows validation results such as loss and accuracy. Step 5 loops back to step 1 to show tuning: people or search tools change the hyperparameters and run training again. The main message is that hyperparameters guide training from outside, while parameters are learned inside the model.

With reduced motion: A static five-box sequence with a printed loop arrow from validation results back to chosen hyperparameters. Colour and labels, not animation, show the distinction between chosen settings and learned parameters.

When a model is trained, two kinds of numbers matter. First are parameters: values adjusted by the training process itself. Second are hyperparameters: settings that people choose, or that a search procedure tries, to control how training happens.

Examples include the learning rate, batch size, number of training rounds called epochs, network depth, and regularisation strength. These settings affect speed, stability, cost and final quality. They do not directly store what the model has learned from the data.

What goes in is a training method plus chosen settings. Then training runs: the system measures error with a loss function, computes updates, and changes the model parameters. What comes out is a trained model and performance results, such as accuracy or loss on validation data.

Hyperparameters are often tuned by comparing several training runs. If the learning rate is too high, training may jump around and fail to settle. If regularisation is too strong, the model may become too simple. Good hyperparameter choices help a model improve reliably, but they do not guarantee good results on their own.

This term is often confused with learning rate because learning rate is one important hyperparameter, not the whole category. It is also confused with parameter, but parameters are learned inside the model, while hyperparameters guide the learning process from outside it.

How “Setting up a document scanner for form matching” maps onto the real thing

In the story	In the AI system
camera resolution, brightness threshold, and matching strictness chosen before the main run	hyperparameters chosen to control training or model behaviour
internal matching values adjusted from examples	model parameters learned from data
test batch used to compare setups	validation data used for hyperparameter tuning
setup with the fewest mismatches	hyperparameter setting that gives better validation performance

A worked example

An information management officer at a regional hub is testing a classifier that routes public technical enquiries to the appropriate unit before the monthly monitoring report is cleared.

Starting from: The same approved, non-sensitive training and validation data are used for each run. Only the learning rate and weight decay are changed.

The officer fixes the training data, validation data and evaluation measure so that the runs can be compared fairly.
Run A uses a high learning rate and low weight decay. Training loss falls, but validation loss fluctuates, indicating unstable fitting.
Run B uses a lower learning rate with the same weight decay. Training is stable and it produces the lowest validation loss of the tested configurations.
Run C keeps the lower learning rate but increases weight decay. Both training and validation performance weaken, indicating that the constraint is too strong for this dataset.
The officer records each configuration and result, selects Run B's hyperparameters, and confirms the resulting model once on an untouched test set.

Result: The learning rate and weight decay are selected hyperparameters. They shaped how training proceeded but were not learned as model weights from the examples.

In a United Nations workplace

A reporting officer in a specialised agency technical division is drafting the evaluation note before the panel reconvenes on three bids for a document-classification service. One bidder describes tuning the learning rate, batch size and weight decay; another presents only its final performance. The officer asks the evaluators to distinguish these chosen training settings from the model weights learned during training and to record whether the bids used comparable validation methods.

Hyperparameters often appear in bid documentation as evidence that a model was systematically tuned. They help evaluators assess whether performance claims are reproducible and fairly compared, rather than the product of one favourable run.

Before you try thisBid contents and panel deliberations are procurement-sensitive. They must not be uploaded to a public or otherwise unapproved AI service. Any testing must use an authorised environment, agreed test data, documented configurations and the evaluation criteria already established for the procurement.

Where you may hear it

A vendor demo during a procurement evaluation meeting
A model review before an internal tool moves from pilot to production
A technical debrief on validation results
An ICT governance board reviewing model documentation

Avoid confusion

Often confused with

Common misconceptions

Common misconception: Hyperparameters are just the numbers the model learns from data.
Correction: Those learned numbers are parameters. Hyperparameters sit one level outside the learned model: they shape the training process or model structure, and are set by people or by a tuning procedure. The difference matters because they are adjusted in different ways and have different effects.
Common misconception: A learning rate and a hyperparameter mean the same thing.
Correction: Learning rate is one example of a hyperparameter. The category is broader and also includes settings such as batch size, number of epochs, dropout rate, weight decay and sometimes architecture choices. Treating one example as the whole category hides important trade-offs.
Common misconception: There is a single best set of hyperparameters for any model.
Correction: Good settings depend on the dataset, model architecture, objective, hardware limits and evaluation method. A setting that works well for one language task, image task or data size may work badly for another. Hyperparameter choices are usually empirical and context-dependent.
Common misconception: If you tune hyperparameters long enough, you are guaranteed a good model.
Correction: Tuning can improve results, but it cannot fix poor data, weak labels, unsuitable model design or a bad evaluation setup. It can also overfit to the validation set if too many trials are used without care. Hyperparameter tuning helps, but it is not magic.

Quick check

Which statement best describes a hyperparameter?

Go deeper

The simple version

Hyperparameters are the controls you set for training, like how big each step should be or how strongly to penalise complexity. The model then learns its own internal values under those controls.

The technical version

In machine learning, hyperparameters are configuration variables that are not directly optimised as part of the model's standard parameter-learning step on the training data. They may define optimisation behaviour, such as learning rate, momentum, batch size and scheduler settings; model capacity or architecture, such as layer count or hidden width; or constraints, such as weight decay, dropout rate and other regularisation terms. A training run takes a particular hyperparameter configuration, minimises a loss over training data to learn parameters, and is then evaluated on validation data. Hyperparameter tuning compares configurations, often by grid search, random search, Bayesian optimisation or adaptive methods, to improve validation performance, robustness or efficiency. Some border cases are context-dependent: for example, in meta-learning or differentiable architecture search, values usually treated as hyperparameters may themselves be optimised by higher-level procedures.

Reviewed external resources

EncyclopaediaHyperparameter (opens in a new tab)Wikipedia
This Wikipedia page gives a broad reference-style overview of the term and suits readers who want a quick definition with related concepts.
Intermediatelink checked 2026-07-31
CourseReducing Loss (opens in a new tab)Google
This course material suits readers who want a practical introduction to reducing loss and understanding how training choices influence results.
Intermediatelink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This visual video helps readers who learn best by intuition and pictures, especially if they want to see how training settings affect neural-network learning at a high level.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Hyperparameter (opens in a new tab)YouTube search
This YouTube search results page helps readers browse multiple video explanations of hyperparameters and suits people comparing teaching styles; it is a search page, not a single vetted source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Hyperparameter (opens in a new tab)Reddit search
This Reddit search results page helps readers see common questions and informal explanations from a community discussion; it is a search page, not a single authoritative source.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

Hyperparameters are the settings that steer learning; parameters are the values the model learns.