Learn AI /Responsible AI, Human Rights and Safety /Algorithmic bias
Algorithmic bias
- Starter
- Responsible AI, Human Rights and Safety
- Human rights practitioners
- Policy officers
- Programme & data teams
Also called: AI bias, bias in AI, discrimination.
Back to the milestone you came from
In five seconds
What it is
Systematic unfair patterns in data, rules or model outputs can disadvantage some people or groups, especially when tools are used in important decisions.
In plain words
An AI system can treat people unfairly because of the way it was designed, trained, or used, even if no one intended that result.
An everyday example
Two friends use the same voice-to-text feature. It transcribes one person's messages accurately but repeatedly changes words in the other person's accent, forcing that person to correct every message. The tool has no intention, but its training data and design make the burden fall unevenly. Looking only at its average accuracy would hide who is doing the extra work.
A library guide points people to shelves, while many AI systems affect higher-stakes areas such as hiring, welfare, policing, healthcare or access to services. Also, real algorithmic bias can come from optimisation goals, interface design, feedback loops and human over-reliance, not only from bad records. The analogy shows uneven performance, but it does not capture the legal tests for discrimination or the full complexity of protected characteristics and context.
Remember thisGood average performance can conceal systematic harm. Examine the data, labels, proxy variables and thresholds, compare errors across relevant groups, and keep people accountable for consequential decisions.
This term is contested
The general concern is widely recognised, but the exact boundary between 'bias', 'fairness' and 'discrimination', and the best way to measure them, is contested and context-dependent.
Understand it
Understand it
Why it matters
Bias matters because AI tools can shape access, attention and opportunity without making the risk obvious. A system may look efficient overall while still failing certain groups more often. For a non-specialist, the key question is not only 'Does it work?' but also 'For whom does it work, who is missed, and who checks the result?' In public services, humanitarian work, hiring, education, health, finance or online platforms, unequal errors can create real harm, weaken trust, and expose an organisation to legal, ethical and reputational risk. Good practice means testing for uneven impacts, documenting limits, keeping meaningful human oversight, and being ready not to use the tool when the risk is too high.
How unfairness can enter an AI system
Show that harmful bias can enter before, during and after a model is built, and that the output can create unequal impacts even if the same rule is applied to everyone.
01
1. Define the task
People choose what problem to solve, what categories to use, and what success means. A poor framing can build unfairness in from the start.
Draw a rounded rectangle labelled 'Task definition'. Under it, add three small tags: 'goal', 'categories', 'success measure'. Add a warning icon with text 'Can exclude important context'.
02
2. Gather data
The examples and labels may be incomplete, outdated, or reflect past inequality. Some groups may appear less often or be described less accurately.
Draw the next rectangle labelled 'Data and labels'. Show small document icons entering it. Add notes: 'missing groups', 'historical patterns', 'inconsistent labels'.
03
3. Build the system
Rules or learned patterns are fitted to the data. Optimising for one metric, such as overall accuracy, can hide worse performance for a smaller group.
Draw a rectangle labelled 'Model or rules'. Add a gear icon. Beneath it, add two mini bars labelled 'overall accuracy high' and 'group error uneven'.
04
4. Use the output
The system produces scores, rankings, summaries or text. If people trust these too much, unfair patterns can affect real decisions and services.
Draw a rectangle labelled 'Outputs in use'. Show icons for a score, a ranked list and a text summary. Add a person icon reviewing them with a note 'human oversight needed'.
05
5. Real-world impact
Different groups may face different error rates, barriers or harms. This is where fairness, non-discrimination and accountability questions become concrete.
Draw a final rectangle labelled 'Impact'. Split it into two side-by-side silhouettes or user groups. Show one with 'accurate, timely' and the other with 'missed, delayed, mislabelled'. Use neutral colours and avoid implying any specific nationality or identity.
A left-to-right flow with four boxes: problem framing, data and labels, model or rules, and real-world use. Under each box, short notes show one way unfairness can enter. A final box shows outputs leading to different impacts for different groups. Read from left to right.
Hovering or tapping each stage reveals one example question: 'Who is missing?', 'Who labels this?', 'What metric is optimised?', 'Who reviews it?', 'Who is harmed if it is wrong?'
Describe this without the diagram
The diagram is a five-stage flow from left to right. Stage one is task definition: unfairness can begin when people choose a narrow goal, unsuitable categories, or a success measure that ignores some groups. Stage two is data and labels: records may be missing, outdated, or shaped by past inequality. Stage three is model or rules: a system can be tuned to perform well on average while performing worse for a smaller group. Stage four is outputs in use: scores, rankings or summaries may influence staff, especially if users trust the tool too much. Stage five is impact: different groups can experience different errors, delays or exclusions. The message is that bias can enter at multiple points in the whole system, not just inside the model.
With reduced motion: A static horizontal flowchart with five labelled boxes and arrows between them: task definition, data and labels, model or rules, outputs in use, and impact. Each box includes one short note about how unfairness can arise at that stage.
People use several related words here, and they are not exactly the same. “Algorithmic bias” usually means unfair patterns linked to a computational system. “AI bias” and “bias in AI” are broader everyday labels for similar problems in AI tools. “Discrimination” is a legal and human-rights term about harmful unequal treatment; some biased systems may amount to discrimination, but not every technical bias question is automatically a legal finding.
What goes in? Usually data, design choices, categories, rules, and goals. A system may be trained on past examples, or it may follow hand-written instructions. If past records under-represent some groups, contain historical inequalities, or use poor labels, the system starts from a distorted picture.
What happens? The system looks for patterns and applies them to new cases. Bias can enter at many stages: how a problem is framed, which data are collected, whose language is recognised well, what counts as “success”, and how results are interpreted. Even a model with the same rule for everyone can still produce unequal harm if the starting data or context are unequal.
What comes out? Scores, rankings, labels, summaries, recommendations, or generated text. These outputs can work less well for some groups, exclude people, or push staff towards unfair choices. This is different from “statistical bias”, which can simply mean a systematic error in estimation, and different again from the “bias” term inside some neural networks, which is a mathematical parameter, not a fairness problem.
The key point is practical: bias is not only inside the model. It can come from the full system around it, including data collection, deployment, user behaviour, and weak human oversight.
How “Voice-to-text that works better for one speaker” maps onto the real thing
| In the story | In the AI system |
|---|---|
| old borrowing records and shelf maps | training data or source data used to build a system |
| books in some languages stored inconsistently | historical inequality or poor data quality affecting some groups more than others |
| digital shelf guide directions | model outputs such as predictions, rankings or recommendations |
| visitor repeatedly sent to the wrong floor | unequal errors or harms experienced by affected users |
A worked example
A case-allocation officer at a municipal housing support desk reviews an automatically generated priority list before the next morning's allocation meeting.
Starting from: Past applications disproportionately reflect people with reliable internet access and complete digital records. Applications received through other channels often contain less structured documentation.
- Testing uses minimised data in an approved environment. Housing case files must not be uploaded to a public AI service or any tool not approved for such personal data.
- The model learns that complete digital documentation is associated with cases that previously received quicker responses, although documentation quality does not reliably indicate urgency.
- Reviewers compare missed urgent cases across application channels and find that the model performs worse on less structured records.
- A single cut-off score places more urgent applicants with incomplete records lower on the priority list. Overall accuracy had concealed the uneven error rate.
Result: The ranking creates a systematic risk of delay for applicants with less complete digital records. The desk pauses deployment, reviews the labels and proxy variables, tests alternative thresholds and retains accountable human review of every priority decision.
In a United Nations workplace
A budget officer at a shared services centre is the finance member of a panel reviewing applications for a temporary staffing roster. The assessment matrix must reach the chief of section for clearance by close of business. With several dozen files still to read, the panel is offered a tool that ranks applicants against the agreed profile.
Previous panel scores are not neutral ground truth. Career breaks, translated applications, unfamiliar institutions and non-standard job titles may have influenced earlier decisions and can become proxies for suitability. The tool may therefore rank some applicants systematically lower, while a respectable overall agreement rate conceals who is being screened out incorrectly.
Before you try thisApplicant files contain HR-sensitive personal data and must not be entered into a public service or any tool not expressly approved for recruitment data. The panel must not use the tool to infer sensitive characteristics or make it the sole basis for exclusion. Any comparison across groups must be lawful and appropriately safeguarded. Record threshold changes, retain accountable panel review and preserve a route to challenge the decision.
Where you may hear it
- A selection panel reviewing an AI-assisted shortlist before signing the assessment matrix
- An ICT governance board during a vendor demonstration, before a pilot is cleared
- An evaluation office presenting error rates by language to the front office
- A Human Rights Council side event on due diligence for automated decisions
Avoid confusion
Avoid confusion
Often confused with
Bias (statistical)
Statistical bias is a systematic gap between a model’s expected prediction across possible training samples and the true target, often caused by restrictive assumptions.
Bias (in a neural network)
A bias is an extra adjustable number added to a neuron's weighted input so the neuron's output can shift even when all inputs are zero.
Fairness
Designing and checking AI systems so they do not produce avoidable, unjust differences in how people or groups are treated or affected.
Automation bias
The tendency to trust a system’s output too much and miss errors that a careful human review might have caught.
Side-by-side
Common misconceptions
Common misconception: If the system does not use a protected characteristic directly, it cannot be biased.
Correction: A system can still produce unfair outcomes through proxy variables, such as postcode, school, language pattern or browsing behaviour, that correlate with protected characteristics. Bias can also come from missing groups, poor labels, or unequal error rates even when the same formal rule is applied to everyone.
Common misconception: Bias only comes from bad data.
Correction: Data are a major source, but not the only one. Problem framing, category design, optimisation targets, thresholds, interface design, deployment context and user over-reliance can all create or worsen unequal impacts. Looking only at the dataset can miss important causes.
Common misconception: High overall accuracy means the system is fair.
Correction: A single average score can hide big performance gaps. If one large group gets very accurate results and a smaller group gets many errors, the overall number may still look strong. Fairness checks often require comparing outcomes and error rates across relevant groups and contexts.
Quick check
Which statement best captures algorithmic bias?
Algorithmic bias is a systematic unfair pattern linked to a computational system. It can arise from data, labels, goals, thresholds or deployment without intentional prejudice. A single error or any group difference is not enough. It differs from statistical bias, automation bias and a model's bias parameter.
Go deeper
Go deeper
The simple version
An AI system can treat people unfairly because of the way it was designed, trained, or used, even if no one intended that result.
The technical version
Algorithmic bias refers to systematic patterns in a computational system that produce unfairly different performance, outcomes or burdens across people or groups. The source may be unrepresentative data, label bias, measurement error, historical inequity, proxy variables, objective-function choices, threshold setting, feedback loops, interface effects or deployment context. In practice, people also say 'AI bias' or 'bias in AI' for this broader class of problems. 'Discrimination' overlaps but is not identical: it is a legal and human-rights concept concerning unjustified unequal treatment or impact, assessed under applicable law and context. The term should also be distinguished from 'statistical bias', which often means systematic deviation in estimation, and from the 'bias' parameter in some machine-learning models, which is a mathematical component rather than a fairness judgement.
Reviewed external resources
Primary sourceRecommendation on the Ethics of Artificial Intelligence (opens in a new tab)UNESCO
This link provides an official international policy framework on AI ethics, useful for policy, governance and organisational leadership audiences.
Intermediatelink checked 2026-07-31
Primary sourceB-Tech Project (opens in a new tab)UN Human Rights Office (OHCHR)
This link offers UN human rights material on business and technology, suited to readers who want a rights-based perspective on risks, responsibility and due diligence.
Intermediatelink checked 2026-07-31
EncyclopaediaAlgorithmic bias (opens in a new tab)Wikipedia
This link gives a broad introductory overview of the term and related ideas, which suits readers who want a quick starting point before going deeper.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Algorithmic bias (opens in a new tab)YouTube search
This link is a search results page for videos on the topic, helpful for readers who prefer visual explanations and want to compare several teaching styles.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Algorithmic bias (opens in a new tab)Reddit search
This link is a search results page for community discussion, useful for seeing common questions, examples and debates, but not as a source of verified facts.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Good average performance can conceal systematic harm. Examine the data, labels, proxy variables and thresholds, compare errors across relevant groups, and keep people accountable for consequential decisions.