Learn AI /Model Quality and Evaluation /Confusion matrix
Confusion matrix
- Intermediate
- Model Quality and Evaluation
- Technical readers
- Programme & data teams
Also called: error matrix.
Back to the milestone you came from
In five seconds
What it is
A table that compares a classifier’s predictions with the real outcomes, so you can see which kinds of mistakes it makes and how often.
In plain words
It is a score table for a sorting system. It tells you how many items were sorted correctly, how many false alarms it raised, and how many real cases it missed.
An everyday example
A team receives 200 survey comments after a public service event in Arusha, Vienna and Copenhagen. One staff member has already reviewed them and marked which comments truly need follow-up. A simple tool then sorts the same comments into “needs action” or “no action needed”. The team makes a four-box tally: comments both the reviewer and tool marked for action, comments both said needed no action, comments the tool flagged by mistake, and comments the tool missed.
The story suggests one human review gives the unquestionable truth. In real evaluation, the “true” label may itself be uncertain, disputed or based on several reviewers. Also, real confusion matrices can cover more than two classes, not just a yes/no split.
Remember thisA confusion matrix is not a single score. It shows the count and direction of each kind of classification error, including which classes a multiclass model confuses.
Understand it
Understand it
Why it matters
A confusion matrix helps people ask a better question than “What is the score?” It shows what kinds of mistakes a system makes, not just how often it is right overall. That matters when different errors have different costs. In a screening task, missing a real case may be worse than raising an extra alert, or the opposite may be true. The table also helps spot class imbalance, where one category is much rarer than another and a high accuracy score can be misleading. For non-specialists making procurement, deployment or oversight decisions, it is a practical way to check whether a system’s behaviour matches the real operational need.
From known answers to a four-box error table
After viewing this, a reader should be able to explain how a confusion matrix is built and what each box says about correct and incorrect predictions.
01
Start with checked examples
Use a test set where each item already has a trusted real label, such as positive or negative.
Draw a stack of cards on the left labelled 'test set'. Each card shows a short item and a small tag 'real label'.
02
Run the classifier
The system predicts a label for each item in the test set.
Draw an arrow from the cards into a middle box labelled 'classifier'. Add output tags reading 'predicted label'.
03
Compare real and predicted labels
For each item, look at the real label and the predicted label together.
Draw paired label chips under the classifier output, one chip marked 'real', one marked 'predicted', repeated for several sample items.
04
Add counts to the right box
If both labels are positive, count a true positive. If both are negative, count a true negative. The two mismatch boxes are false positive and false negative.
On the right, draw a 2×2 grid. Rows: 'real positive', 'real negative'. Columns: 'predicted positive', 'predicted negative'. Fill cells with labels TP, FN, FP, TN and example counts.
05
Read the error pattern
Use the table to see whether the system mainly misses real cases, creates false alarms, or performs well on both.
Add callout notes pointing to FN as 'missed cases' and FP as 'false alarms', with a small footer note 'used to calculate accuracy, precision, recall'.
Show a left-to-right flow: labelled test items enter a classifier, then counts are placed into a 2×2 table with rows for real labels and columns for predicted labels. Read the table by asking, for each item, what the real label was and what the system predicted, then add one count to the matching box.
Hover or tap each cell to highlight the corresponding real/predicted combination and show a plain-language label such as 'missed positive case' or 'correctly rejected negative case'.
Describe this without the diagram
The diagram shows how a confusion matrix is made. First, begin with a test set of items that already have trusted real labels. Second, pass those items through a classifier that predicts labels. Third, compare each item’s real label with its predicted label. Fourth, place each item into one of four boxes in a two-by-two table: true positive for real positive and predicted positive, false negative for real positive and predicted negative, false positive for real negative and predicted positive, and true negative for real negative and predicted negative. The purpose is to reveal the pattern of correct results, missed cases and false alarms.
With reduced motion: A static left-to-right diagram shows test items with real labels, a classifier, and a 2×2 table with rows for real labels and columns for predicted labels. Arrows connect the stages, and each cell is labelled true positive, false negative, false positive or true negative.
A confusion matrix is a small table used to check a classification system. A classification system sorts items into labels, such as “relevant” or “not relevant”. The table compares two things for each item: the real label and the system’s predicted label.
For a simple two-class task, the table has four main counts. True positives are items correctly predicted as positive. True negatives are items correctly predicted as negative. False positives are items wrongly predicted as positive. False negatives are items wrongly predicted as negative.
What goes in is a set of examples where the real answer is already known. What happens is that you run the system on those examples and count how many cases fall into each box. What comes out is not one single score, but a picture of the pattern of errors.
That matters because one overall score, such as accuracy, can hide important problems. A system may look good overall but still miss many important cases, or raise too many false alarms. The confusion matrix helps you calculate other measures such as precision and recall, and helps you judge whether the system fits the real use case.
How “Sorting survey messages into “needs action” or “no action needed”” maps onto the real thing
| In the story | In the AI system |
|---|---|
| survey comments with known follow-up labels | examples with true class labels |
| tool sorts comments into two groups | classifier makes predicted labels |
| four-box tally of matches and mismatches | confusion matrix counts |
| mistaken flags and missed issues | false positives and false negatives |
A worked example
The communications officer evaluates the classifier on 120 files already labelled by the records focal point: 40 final public products, 40 working drafts and 40 duplicates.
Starting from: For actual final products, the predictions are 35 final, 4 draft and 1 duplicate. For actual drafts, they are 6 final, 30 draft and 4 duplicate. For actual duplicates, they are 2 final, 3 draft and 35 duplicate.
- Create a 3×3 table with actual labels as rows and predicted labels as columns.
- Enter each count in the corresponding cell. The diagonal contains 35 correctly identified final products, 30 correctly identified drafts and 35 correctly identified duplicates.
- Inspect the off-diagonal cells. The largest error is the six working drafts predicted to be final public products.
- Add the diagonal counts to obtain 100 correct predictions out of 120. Use the individual rows and columns to calculate per-class recall and precision if needed.
Result: Overall accuracy is 100 out of 120, or 83.3%. The confusion matrix adds the operationally useful finding that six drafts were labelled as final products. That error may warrant closer review even though the classifier performs equally well on final products and duplicates, correctly identifying 35 of 40 in each class.
In a United Nations workplace
A communications officer in an office working in English, French and Arabic is preparing shared communications files for a records retention review due at close of business. The officer tests a classifier that proposes one of three labels: final public product, working draft or duplicate.
A confusion matrix shows not only how many labels match the records focal point's review, but which categories the classifier confuses. Drafts labelled as final products create a different problem from duplicates labelled as drafts, even if both count equally in an overall accuracy score.
Before you try thisThe officer must not upload shared-drive files to a tool that has not been approved for confidential or personal information. Evaluation should use an approved environment and minimised copies. The classifier may support the review, but a designated colleague must make retention decisions.
Where you may hear it
- During an ICT governance board's first reading of a model evaluation report
- At a vendor demonstration convened by a procurement unit
- In a quality assurance meeting before a classifier is uploaded to the portal
- During clearance of an evaluation office's findings on an automated tool
Avoid confusion
Avoid confusion
Often confused with
False positives and false negatives
False positives are wrong yes predictions, while false negatives are wrong no predictions.
Accuracy
The share of cases a system labels correctly out of all the cases it was tested on.
Precision
The share of flagged results that are actually correct positives.
Recall
The share of real positive cases that a system successfully flags.
Common misconceptions
Common misconception: A confusion matrix is just another name for accuracy.
Correction: Accuracy is only one number you can calculate from the table. The matrix itself keeps the four underlying counts separate, so you can see whether errors come from false alarms, missed cases, or both.
Common misconception: If the diagonal boxes are large, the model is good enough.
Correction: Large correct-count boxes can still hide a serious problem if one class is much more common than the other. For example, a model can look strong overall while missing many rare but important positive cases.
Common misconception: The confusion matrix only matters for technical specialists.
Correction: It is useful for anyone who must judge whether a system is safe and useful in practice. Non-technical decision-makers often need to know which error type matters more in the real workflow, and the matrix shows exactly that.
Common misconception: A confusion matrix and false positives/false negatives are the same thing.
Correction: False positives and false negatives are two boxes inside the matrix. The confusion matrix is the full table, including correct predictions as well as both kinds of mistakes.
Quick check
Why might a confusion matrix be more informative than accuracy alone?
Accuracy compresses performance into one number. A confusion matrix keeps the counts for correct positives, correct negatives, false positives and false negatives separate, so you can judge the real error pattern and whether it fits the use case.
Go deeper
Go deeper
The simple version
It is a score table for a sorting system. It tells you how many items were sorted correctly, how many false alarms it raised, and how many real cases it missed.
The technical version
For a classifier evaluated on labelled test data, a confusion matrix tabulates counts by actual class versus predicted class. In binary classification, the four cells are true positive, false positive, false negative and true negative. In multiclass classification, the matrix extends to one row and one column per class, and off-diagonal cells show which classes are being confused with which others. The matrix is not itself a single metric. Instead, it is the count structure from which metrics such as accuracy, precision, recall, specificity and F1 score can be derived. Its usefulness depends on sound evaluation design: representative test data, clear label definitions, separation from training data, and awareness of class imbalance and error costs.
Reviewed external resources
Primary sourceAI Risk Management Framework (opens in a new tab)NIST
This official framework link helps readers place model evaluation within wider AI risk management and oversight, and suits programme leads and governance-focused readers.
Intermediatelink checked 2026-07-31
EncyclopaediaConfusion matrix (opens in a new tab)Wikipedia
This link gives a broad reference-style overview of the concept, terminology and variants, and suits readers who want a general starting point.
Intermediatelink checked 2026-07-31
CourseClassification: Accuracy, recall, precision (opens in a new tab)Google
This course link helps readers connect the table to accuracy, precision and recall, and suits beginners moving into practical evaluation.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Confusion matrix (opens in a new tab)YouTube search
This is a search results page for video explanations on the topic, useful for readers who learn visually and want to compare several teaching styles.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Confusion matrix (opens in a new tab)Reddit search
This is a search results page for community discussions about the topic, useful for readers who want informal explanations, questions and practical troubleshooting.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
A confusion matrix is not a single score. It shows the count and direction of each kind of classification error, including which classes a multiclass model confuses.