Learn AI / Term Explorer /Model Quality and Evaluation
Model Quality and Evaluation
How anyone can tell whether a system actually works, and the many ways a good-looking score can mislead.
Starter
Accuracy
The share of cases a system labels correctly out of all the cases it was tested on.
Evaluation
Evaluation checks how well a system performs on a defined task, while metrics are the specific measures and evals are the tests used to produce them.
False positives and false negatives
False positives are wrong yes predictions, while false negatives are wrong no predictions.
Overfitting
A model fits its training examples so closely that its performance drops on new cases.
Intermediate
Baseline
A reference method or result used to judge how a model performs against a relevant starting point.
Benchmark
A benchmark is a standard test set-up used to compare how well systems perform on a defined task under stated conditions.
Class imbalance
A dataset has this problem when one class appears far more often than another, which can make overall accuracy look strong while a model misses rare cases.
Confusion matrix
A table that compares a classifier’s predictions with the real outcomes, so you can see which kinds of mistakes it makes and how often.
Data leakage
Data leakage happens when future, target or held-out test information improperly influences model training or evaluation, making performance look better than it is.
Generalisation
The ability of a model to give useful results on new cases, not just on the examples used to build or tune it.
Memorisation
Memorisation happens when a model encodes information specific to individual training examples or small subsets of them.
Precision
The share of flagged results that are actually correct positives.
Recall
The share of real positive cases that a system successfully flags.
Reliability
Reliability is how consistently a system gives similar quality results when it is used again on comparable cases and conditions.
Robustness
The ability of a model or system to keep performing acceptably when inputs, conditions or small disturbances change.
Underfitting
A model misses important patterns because it is too simple, too constrained, or too weakly trained for the task.
Deeper
Bias (statistical)
Statistical bias is a systematic gap between a model’s expected prediction across possible training samples and the true target, often caused by restrictive assumptions.
Calibration
Calibration describes how closely a system’s probability scores match the observed frequencies of the corresponding outcomes.
Cross-validation
A method for estimating how well a model will perform on new data by repeatedly training on one part of a dataset and checking results on another part.
F1 score
A single score that balances how many positive cases a system finds with how often its positive predictions are correct.
Variance
Variance describes how much a fitted model’s predictions change when the same learning procedure is trained on slightly different samples.