Learn AI /Data and Infrastructure /Annotation and data labelling
Annotation and data labelling
- Starter
- Data and Infrastructure
- Human rights practitioners
- Programme & data teams
Also called: data labelling, labelling, data annotation, annotators.
Back to the milestone you came from
In five seconds
What it is
People or software tools add task-relevant tags, notes or boundaries to data for model training, testing, analysis or monitoring.
In plain words
People look at data and add useful tags or markings so an AI system has clearer examples to learn from or be tested against.
An everyday example
A city engineering team prints 200 satellite photos after a storm. They agree on a guide: draw a red outline around road sections that look damaged, add a yellow mark for uncertain cases, and leave intact roads unmarked. Different staff members mark the photos, and a supervisor checks samples where people disagreed. At the end, the team has a stack of photos with consistent markings that can be used to train a system to spot likely road damage faster.
Real annotation projects are usually more complex than marking paper photos. The rules may involve many classes, edge cases and formats, and the data may be text, audio or video rather than images. Also, model training does not copy markings directly like a person reading the paper; it uses many examples to adjust internal parameters statistically, so even carefully labelled data does not guarantee correct outputs.
Remember thisAnnotation turns raw material into task-specific examples. Clear categories, consistent instructions and documented review matter more than a large pile of unexplained labels.
Understand it
Understand it
Why it matters
If you hear that an AI system was “trained on data”, annotation helps explain what that often means in practice: people or tools turned messy inputs into examples with clear targets. The quality of those targets affects accuracy, fairness, safety and cost. Poorly designed labels can hide important differences, force sensitive realities into crude categories, or reproduce bias from the people and rules used to create them. For non-specialists, this matters when choosing vendors, reviewing evidence, setting procurement requirements, or deciding whether a system is suitable for a high-stakes task. Asking who labelled the data, under what rules, with what checks, is often more useful than asking only which model was used.
How annotation turns raw data into training-ready data
Show the ordered process from raw data to reviewed labels, and explain the difference between simple labelling and broader annotation.
01
1. Raw data arrives
Start with unlabelled items such as images, text, audio or video.
Show mixed icons for an image, a document, a waveform and a video frame, all with pale grey placeholders and no markings.
02
2. Rules are defined
The team chooses the classes or mark-up needed and writes instructions for annotators.
Show a guideline sheet with simple examples: 'damaged road', 'uncertain', 'not damaged' and a note saying 'draw boundary if visible'.
03
3. Data is marked
Annotators or tools add labels, boxes, highlights, timestamps or other tags to each item.
Show the same sample items now with a tag on text, a box on an image, and a timestamp marker on audio.
04
4. Quality is checked
Disagreements, missing marks and unclear cases are reviewed so the dataset is more consistent.
Show two versions of one image side by side with a small warning icon, then a merged reviewed version with a tick.
05
5. Labelled dataset is produced
The result is a more structured dataset that can support training, testing or monitoring.
Show a table linked to sample files, with columns for item ID, label, annotation file and review status.
A left-to-right step diagram with five boxes. Read from raw data, to task design, to human or tool marking, to quality review, to the finished labelled dataset. Each box includes a tiny example of what changes at that stage.
Hover or tap each step to reveal a short example of the annotation format used there, such as class label, bounding box, highlighted text span or timestamp.
Describe this without the diagram
This diagram explains annotation as a five-step process. First, there is raw unlabelled data such as images, text, audio and video. Second, a team defines rules, including what labels to use and how to mark unclear cases. Third, annotators or software tools add labels or richer annotations such as boxes, highlights or timestamps. Fourth, the work is checked for disagreements and mistakes, and unclear items are reviewed. Fifth, the result is a labelled dataset that is more structured and can be used for training, testing or monitoring a model. The key message is that annotation is broader than simple labelling, and that quality review matters.
With reduced motion: A static five-box horizontal process graphic with arrows between boxes, showing raw data, rules, marking, review and finished labelled dataset, with no animation or hover-only content.
Annotation and data labelling are closely related, but they are not always exactly the same. Data labelling usually means attaching a target tag such as “damaged road” or “not damaged road”. Annotation is broader: it can also mean drawing boxes around objects, marking sentence parts, highlighting spans of text, or adding notes about what appears in an image, audio clip or document. Annotators are the people, or sometimes software tools, that do this work.
What goes in can be many kinds of data: images, text, speech, video or tables. A team first decides what labels or annotations are needed and writes rules so different people apply them in a similar way. Then annotators review each item and add the required mark-up. Quality checks may compare different annotators’ work, measure agreement, and correct unclear cases.
What comes out is a labelled or annotated dataset. That dataset can be used to train a model, test whether it performs well, or monitor whether it is still working acceptably. The output may look structured, such as a spreadsheet column of labels, or more detailed, such as polygons around flooded areas on a map or highlighted names in a document.
This is often confused with data cleaning and with ground truth. Data cleaning fixes formatting errors, duplicates or broken records; annotation adds meaningful tags or marks. Ground truth usually means the best available reference answer for evaluation or training, while labels are the actual tags people assigned. In practice, some labels become part of what a project treats as ground truth, but they are not automatically perfect.
How “Marking damage on printed satellite photos” maps onto the real thing
| In the story | In the AI system |
|---|---|
| printed satellite photos | raw input data such as images |
| red outlines and yellow marks | labels and annotations added to the data |
| the written guide for staff | annotation instructions or labelling guidelines |
| the supervisor checking disagreements | quality control and agreement review |
A worked example
A reporting officer in a Resident Coordinator’s Office is preparing the first draft of a progress report and wants a text model to locate stated actions in publicly issued meeting summaries.
Starting from: Publicly issued meeting summaries with their headings and body text preserved, plus a draft annotation guide.
- Define span-level tags for the action, responsible entity and timeframe, together with a document-level label for summaries containing no stated action.
- Have annotators independently mark the same calibration set, compare disagreements and revise the guide where the boundary or meaning of a tag is unclear.
- Label the remaining summaries, flag uncertain passages for adjudication and record the final decision rather than silently overwriting the original annotation.
- Create separate training, validation and test sets, ensuring that duplicate or near-duplicate summaries do not appear across them.
Result: The result is a versioned annotated corpus in which relevant text spans and no-action cases can be traced to the guide and review decision. It can support model training and held-out testing, with any model output still reviewed before it enters the progress report.
In a United Nations workplace
Before close of business on the last day of a rotation, the partnerships officer in a specialised agency’s technical division updates the handover note with the annotation scheme for a dashboard that groups published partner activity summaries by theme. The note records which text receives each label, how mixed-theme items are handled and which examples remain uncertain, so the incoming colleague can continue the work consistently.
Annotation is not merely adding topic names. The officer must define the categories, decide whether an item may receive several labels, document edge cases and review disagreements. A model trained on the dataset will reproduce those decisions, including any inconsistencies.
Before you try thisPublished summaries may be suitable for the task, but raw source files can contain personal, confidential or commercially sensitive information. Such material must not be copied into a labelling platform or external AI tool unless it is approved for that class of data. Remove unnecessary personal data, restrict access and retain the source and review history for each label.
Where you may hear it
- A vendor demo during a procurement meeting
- A first reading of an annotation guide in an inter-agency working group
- A data quality review before a model is tested
- An ICT governance board discussion on training data
Avoid confusion
Avoid confusion
Often confused with
Data cleaning
The process of checking data, fixing problems, and putting it into a usable form for analysis or model training.
Ground truth
The checked reference answer used to train or evaluate a model, so its output can be compared with the best available real-world answer.
Metadata
Information that describes other data, such as who created it, when, where, in what format, and under what rules it can be used.
Structured data
Information organised into predefined fields, often in rows and columns, so software can process it predictably.
Common misconceptions
Common misconception: Labelling just means putting one tag on each file.
Correction: Sometimes it does, but annotation can be much richer. A project may need boxes around objects, highlighted words in text, speaker turns in audio, timestamps in video, or notes about uncertainty. Treating all of this as a single simple tag can hide real differences in workload, quality needs and risk.
Common misconception: If humans labelled the data, the labels are automatically correct.
Correction: Human labels can still be inconsistent, rushed, culturally narrow or based on unclear instructions. Two qualified annotators may disagree on borderline cases. That is why projects use guidelines, sampling, adjudication and agreement checks instead of assuming that one human answer is always the truth.
Common misconception: Annotation and data cleaning are the same job.
Correction: They often happen in the same pipeline, but they solve different problems. Data cleaning fixes issues like duplicates, broken formatting or impossible values. Annotation adds meaning for a task, such as category tags or marked regions. A clean dataset can still be unlabelled, and a labelled dataset can still need cleaning.
Common misconception: More labelled data always solves performance problems.
Correction: More data can help, but only if the labels fit the real task and the examples represent the situations the model will face. If the instructions are weak, the categories are poorly chosen, or the data distribution is narrow, adding more of the same may simply scale up the same problem.
Quick check
Which statement best describes annotation and data labelling?
Both terms concern adding task-relevant information to data for training, testing or review. A label may assign one class to each item. Depending on the project, annotation may also include boxes, highlights, timestamps and other mark-up. This is different from data cleaning, metadata and ground truth, and human-created labels still need quality checks.
Go deeper
Go deeper
The simple version
People look at data and add useful tags or markings so an AI system has clearer examples to learn from or be tested against.
The technical version
Annotation is the process of attaching task-relevant information to raw data instances. In classification, this may be a discrete class label. In other tasks, it may be token-level tags in text, bounding boxes or polygons in images, segmentation masks, keypoints, timestamps, speaker labels, rankings or free-text notes. Data labelling usually refers to the simpler end of this range, while annotation is the broader term. The process typically includes schema design, instructions, annotator training, sampling, assignment, quality control, adjudication of disagreements, and storage of labels in a machine-readable format linked to each source item. The resulting annotated corpus can support supervised learning, evaluation, error analysis and ongoing monitoring. Labels may be treated as reference targets, but they remain dependent on the task definition, the annotators, and the governance choices around privacy, labour and data handling.
Reviewed external resources
Primary sourceAI Risk Management Framework (opens in a new tab)NIST
This NIST resource helps readers connect annotation choices to AI risk management and quality controls, which suits managers, reviewers and technical leads.
Intermediatelink checked 2026-07-31
Primary sourceOECD AI Principles (opens in a new tab)OECD
This OECD resource helps readers place annotation within wider trustworthy AI principles and governance concerns, which suits policy and programme audiences.
Intermediatelink checked 2026-07-31
EncyclopaediaData annotation (opens in a new tab)Wikipedia
This Wikipedia link gives a broad overview of data annotation concepts and examples, which suits readers who want a quick general introduction.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Annotation and data labelling (opens in a new tab)YouTube search
This YouTube search results page helps readers find different video explanations and demonstrations of annotation and data labelling, which suits people who learn well by watching but should be read as a search page, not a single vetted source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Annotation and data labelling (opens in a new tab)Reddit search
This Reddit search results page helps readers see common practitioner questions and informal discussions about annotation and data labelling, which suits readers seeking community perspectives but should be read as a search page, not a single authoritative source.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Annotation turns raw material into task-specific examples. Clear categories, consistent instructions and documented review matter more than a large pile of unexplained labels.