presentsUNAIVERSE
The UN AI timeline

    Learn AI /Responsible AI, Human Rights and Safety /Non-discrimination

    Non-discrimination

    Also called: equal treatment.

    Detail

    In five seconds

    What it is

    A principle and legal obligation to prevent unjustified harmful treatment based on protected traits or close proxies for them.

    In plain words

    People should not face unjustified barriers, penalties or worse service because of protected traits, or because a system uses clues that stand in for them.

    An everyday example

    A team receives 500 applications for a training programme and first sorts them by completeness. The checklist says an address must match a small approved list of neighbourhoods and that forms written in one standard style count as easier to process. On paper, the rule looks about completeness, not about people. In practice, it pushes aside many applicants from migrant communities and people who had less help filling in forms, even when they meet the real programme requirements. The team then reviews fewer of those applications, so the first sorting step quietly shapes the final outcome.

    Where the story stops working

    A paper checklist is simpler than most AI systems. Real systems may combine many variables at once, learn patterns from past data and produce different error rates across groups without any single rule being obvious. Also, some differences in treatment can be lawful in limited cases, so the key question is not only whether groups differ, but whether the difference is justified, necessary and proportionate under the relevant rules.

    Remember thisA system can discriminate without naming protected traits if its data, rules or use impose unjustified barriers, burdens or errors linked to them.

    This term is contested

    The core idea is widely recognised, but the exact legal tests, protected categories and permitted exceptions vary across jurisdictions and sectors.

    Understand it

    Understand it

    Why it matters

    Non-discrimination matters because AI systems can shape access to jobs, services, information and scrutiny at scale. If unequal treatment enters early, it can be repeated quickly and hidden behind technical language. For a non-specialist, the key issue is not whether a system looks neutral, but whether it creates unjustified barriers or errors for some groups. This affects legal risk, public trust, human rights and the quality of decisions. It also affects accountability: someone must check the system, justify its use and provide a way to challenge harmful outcomes.

    How a neutral-looking system can still discriminate

    Show how data, rules and thresholds can create unequal outcomes even when a system does not mention a protected trait directly.

    1. 01

      1. Inputs

      The system starts with data, labels and categories. Some fields may be missing, uneven or influenced by past exclusion.

      Draw a box labelled Inputs with small items inside: data, labels, categories, instructions. Add a note: some variables may be proxies such as postcode or school history.

    2. 02

      2. Processing rule

      The system scores, ranks or flags people using patterns from the data or fixed rules.

      Draw a second box labelled Score or rank. Connect from Inputs with an arrow. Show gears or stacked bars to suggest processing, not decoration.

    3. 03

      3. Cut-off

      A threshold turns a score into an action such as review, reject or prioritise.

      Draw a decision gate labelled Cut-off score. Split one arrow into two paths: above threshold and below threshold.

    4. 04

      4. Outcomes by group

      Compare who gets through, who is blocked and who gets errors. Different rates may show a discrimination risk that needs review.

      Draw two grouped outcome panels with simple bar marks for Group A and Group B. Label examples: shortlisted, delayed, false flag, extra checks. Add a warning icon with text: investigate unequal impact.

    A left-to-right flow diagram with four stages: inputs, model or rules, threshold decision and outcomes by group. Read from left to right to see how a seemingly neutral design choice can lead to different error rates or access levels for different groups.

    Hover or tap each stage to reveal one example of how discrimination can enter there: proxy data, biased labels, threshold choice, or unequal error rates.

    Describe this without the diagram

    This diagram explains a four-step process. Step 1 is Inputs: data, labels, categories and staff instructions enter the system, and some fields may work as proxies for protected traits. Step 2 is Processing rule: the system scores, ranks or flags cases using learned patterns or fixed rules. Step 3 is Cut-off: a threshold turns that score into an action, such as review or rejection. Step 4 is Outcomes by group: the final panel compares who is accepted, delayed, wrongly flagged or filtered out across groups. The main lesson is that discrimination can appear even if the system never names a protected trait, because proxy data, labels, rules and thresholds can still produce unequal impacts.

    With reduced motion: Show the same four boxes in a static left-to-right row with arrows between them, plus a final side-by-side outcome panel comparing groups without animation.

    Non-discrimination means people should not face unjustified loss of access, opportunities or equal treatment because of traits such as race, sex, disability, religion, age, nationality or other protected status. In AI systems, this includes direct discrimination, where someone is treated worse because of a protected trait, and indirect discrimination, where an apparently neutral rule disadvantages a protected group without adequate justification.

    What goes in matters. Inputs can include training data, labels, categories, scoring rules, cut-off scores and staff instructions. A cut-off score is the point used to sort cases, such as pass or fail. Systems may also use proxy variables, which are stand-ins like postcode, school history or writing style that can act as indirect indicators of a protected trait.

    What happens next is that the system sorts, predicts, ranks, flags or recommends. At each step, people should ask: who may be disadvantaged, compared with whom, for what reason, and is that difference lawful, necessary and proportionate? This is often confused with fairness. Fairness is broader and can mean different things. Non-discrimination is more closely tied to legal rights and unjustified unequal treatment.

    What comes out may be a hiring shortlist, a risk score, a fraud alert or a service priority label. If one protected group is filtered out more often, gets more errors, or faces extra checks without a valid reason, that may signal discrimination. Reducing the risk usually needs better data, testing across groups, human review, clear accountability and a way for affected people to question the result.

    How “Sorting application folders with a faulty checklist” maps onto the real thing

    In the storyIn the AI system
    Completeness checklistScoring rules or screening criteria
    Approved neighbourhood listProxy variable that can track protected traits
    Forms in one standard styleData pattern that favours one group’s typical behaviour
    Applications pushed aside earlyDisparate impact from an apparently neutral filter

    A worked example

    A staffing focal point in a regional bureau must clear an internship shortlist for the front office by close of business. An approved automated screen ranks applications before human review.

    Starting from: Applications, historical hiring outcomes, education history, employment gaps and writing-style indicators. The system advances only candidates above a fixed score.

    1. The focal point compares advancement rates for applicants using the platform’s standard writing style and those using a second-language style, while checking for similar qualifications.
    2. Applicants using a second-language style advance at a materially lower rate, and manual review does not reveal a corresponding difference in relevant qualifications.
    3. Feature analysis shows that writing style and school-name matching are acting as proxies for language background or national origin rather than measuring the internship criteria.
    4. Reviewers assess whether those distinctions are job-related, lawful and objectively justified. They find that the proxy features are not necessary for the stated selection criteria.
    5. The features are removed, the threshold is retested across groups, and a manual review path is added for borderline cases. Applications must not be uploaded to an unapproved AI service because they contain personal and recruitment-sensitive data.

    Result: The disparity alone did not establish discrimination, but it required investigation. Once the proxy effects lacked an objective justification, the screen could not be relied on as configured. Testing, documented justification and continuing human oversight were needed before further use.

    In a United Nations workplace

    An information management officer in a country office is finishing a handover note before close of business. It records that a translation-and-triage tool used on community feedback is more likely to mark messages in some language variants as unclear, making those entries less likely to reach the priority queue for the next situation report.

    Language variant may correlate closely with protected characteristics. If error rates differ by group, a neutral-looking threshold can produce unjustified disparate impact. The incoming officer needs the test results, the rationale for the threshold and the agreed manual review safeguard, not merely a note that the tool “works overall”.

    Before you try thisCommunity feedback may contain names, contact details, protection concerns or security-sensitive information. It must not be entered into any tool not approved for that material, because disclosure or retention could expose individuals. Until unequal error rates are assessed and mitigated, no message affecting safety or access to assistance should be deprioritised without human review.

    Where you may hear it

    • During clearance of terms of reference for an automated recruitment screen
    • At a vendor demo in a procurement meeting, when error rates by group are requested
    • In an inter-agency working group reviewing a needs assessment tool
    • While an evaluation office examines disparate impacts before a tool is scaled up

    Avoid confusion

    Avoid confusion

    Often confused with

    Common misconceptions

    • If a system does not use race, sex or another protected trait directly, it cannot discriminate.

      A system can still discriminate through proxy variables, uneven data, biased labels or thresholds that create worse outcomes for a protected group. Postcode, school history, device type or writing style can sometimes stand in for protected traits.

    • Non-discrimination just means every person must be treated in exactly the same way.

      Equal treatment is not always identical treatment. In some contexts, removing barriers may require reasonable accommodation or different handling that supports equal access. The main question is whether differences in treatment are unjustified, unlawful or disproportionate.

    • If the model is statistically accurate overall, there is no non-discrimination problem.

      Average accuracy can hide large differences between groups. A system may work well overall but still produce more false alarms, missed cases or exclusions for one group. Group-level testing is therefore important.

    Quick check

    Which case is the clearest warning sign of non-discrimination risk in an AI system?

    Go deeper

    Go deeper

    The simple version

    People should not face unjustified barriers, penalties or worse service because of protected traits, or because a system uses clues that stand in for them.

    The technical version

    In AI governance, non-discrimination means avoiding unjustified differential treatment and unjustified disparate impact linked to protected characteristics or closely correlated proxies. Risk can enter through sampling, labelling, feature selection, optimisation goals, threshold setting, deployment context and unequal error rates. Assessment often asks whether a system causes adverse effects for particular groups, whether the distinction is lawful and objectively justified, and whether safeguards such as impact assessment, human oversight, monitoring and remedy are in place. This is related to fairness but not identical to it: fairness includes broader and sometimes contested criteria, while non-discrimination is more directly grounded in legal and human rights obligations.

    Reviewed external resources

    Where this came from

    Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

    The one thing to remember

    A system can discriminate without naming protected traits if its data, rules or use impose unjustified barriers, burdens or errors linked to them.