Learn AI /Computer Vision and Language Technology /Image captioning
Image captioning
- Intermediate
- Computer Vision and Language Technology
- Communications teams
- Programme & data teams
Also called: alt text generation.
Back to the milestone you came from
In five seconds
What it is
A task that turns an image into a short written description, often to summarise visible content for search, accessibility or workflow support.
In plain words
It looks at a picture and drafts a short description in words.
An everyday example
Imagine describing a photograph to someone who cannot see it. You mention the main visible action and the objects that matter, rather than every colour, shadow and chair. Your description may be useful, but it can still be wrong if you guess who someone is or why they are there. Image captioning does a similar job automatically, so its summary still needs checking against the photograph.
A human colleague can use wider context, ask questions and recognise uncertainty in a richer way. An image captioning system only works from patterns in data and may produce fluent text even when the image is unclear or the description is wrong. Also, some systems generate only short captions, not the fuller context a person might add.
Remember thisImage captioning generates a short description from visual features. It can support search, accessibility and drafting, but it cannot guarantee that identities, relationships or context have been interpreted correctly.
Understand it
Understand it
Why it matters
For non-specialists, image captioning matters because it affects accessibility, search, publishing speed and record-keeping. A good caption can help people who use screen readers, help staff find images later, and reduce repetitive manual work. But poor captions can mislead readers, exclude people if the description is inaccurate, or create reputational and operational problems if sensitive content is described wrongly. In practice, the key decision is not whether the tool sounds impressive. It is whether the captions are reliable enough for your purpose, in the right language, with proper review and safe handling of images.
How an image becomes a caption
After viewing this, a reader should be able to describe the main stages: image in, visual features extracted, language generated, caption checked for fit.
01
1. Input image
The system receives a photograph or other image file.
Draw a rectangular photo placeholder on the left with simple shapes suggesting a truck, boxes and people.
02
2. Visual analysis
The system extracts patterns from the image, such as objects, positions and some relationships.
Show the same thumbnail with small callouts pointing to 'people', 'boxes', 'truck' and 'queue area'.
03
3. Text generation
Using those visual patterns, the system assembles words into a likely description.
Place floating word chips like 'workers', 'unload', 'boxes', 'from', 'truck' moving into a sentence line.
04
4. Caption output
The system outputs a caption, but a user may still need to review it for accuracy, bias or missing context.
Display a sentence in a text box and add a small review icon or warning triangle beside it.
A left-to-right flow diagram with four boxes connected by arrows. The first box shows an image thumbnail. The second shows detected visual cues such as people, boxes and a truck. The third shows word choices forming a sentence. The fourth shows the final caption with a small warning that wording can be incomplete or wrong. Read from left to right.
On hover or tap, each stage reveals one sentence explaining what information is kept and what can be lost before the next stage.
Describe this without the diagram
This diagram shows image captioning as a four-step flow. First, an image enters the system. Second, the system analyses visible patterns such as objects and positions. Third, it converts those patterns into words and arranges them into a sentence. Fourth, it outputs a caption. The final stage notes that the caption may still be incomplete or incorrect, so human review may be needed.
With reduced motion: A static four-column diagram replaces animation. Arrows connect image input, visual analysis, text generation and caption output, with the same labels and explanations visible at once.
Image captioning takes a picture as input and produces a sentence or short paragraph as output. The goal is not just to spot objects, but to describe a scene in words. For example, instead of only saying “boxes” and “people”, it may produce “Two workers stack food boxes beside a truck”.
Many systems do this by combining two parts. One part analyses the image and extracts patterns such as shapes, textures, objects and their positions. The other part generates text, choosing words in an order that fits the visual patterns it found. A newer combined system may do both together.
This is different from image classification, which usually assigns one label such as “market” or “flood”. It is also different from object detection, which marks where items appear, often with rectangles called bounding boxes. Image captioning tries to express the overall scene in language.
The result can be useful, but it is not guaranteed to be correct or complete. A system may miss small details, misidentify people or objects, or add details that are not visible. Caption quality can also vary by language, image type and training data, especially for low-resource languages or less common settings.
How “Describing a photograph over the telephone” maps onto the real thing
| In the story | In the AI system |
|---|---|
| the photo sent to the coordinator | the input image |
| the colleague noticing people, sacks, truck and queue | the visual analysis of objects and relationships |
| the one-line update in the log | the generated caption |
| the daily log being easier to scan | use of captions for accessibility, search or workflow support |
A worked example
A communications officer needs a short caption for a photograph selected for a side-event concept note before it goes to the front office for clearance.
Starting from: A photograph shows four people seated behind nameplates at a long table, one person speaking into a microphone, and a presentation screen displaying a chart.
- The visual component encodes features associated with people, a table, nameplates, a microphone, a screen and a chart.
- It represents likely relationships, including one seated person using the microphone while the others face towards the speaker.
- The language component predicts a short sequence of words conditioned on those visual features, giving priority to the main visible action.
- The officer checks the photograph against the draft, avoids inferring identities or roles from appearance, and edits the wording before clearance.
Result: Possible caption: “A panellist speaks at a table while three other participants listen, with a chart displayed on the screen behind them.”
In a United Nations workplace
At a duty station working in three languages, a communications officer is preparing the photo annex to a delegation briefing before comments arrive from capitals. An approved captioning tool produces first-pass descriptions of images from the session so the officer can select photographs and draft accessible captions.
The captions make the image set searchable and give the officer a starting point for alt text. They describe visible content, not the significance of an exchange, and any versions in other languages require separate linguistic review.
Before you try thisUnpublished, personal or security-sensitive images must not be uploaded to a tool that is not approved for that material. The officer must verify every caption and remove unsupported identities, affiliations, locations or interpretations before the annex is cleared or any image is published.
Where you may hear it
- A website accessibility review before publication
- A front-office clearance of photographs and captions
- A vendor demonstration to an ICT governance board
- A photo-archive meeting on search and metadata
Avoid confusion
Avoid confusion
Often confused with
Image classification
A system assigns one or more predefined labels to an image based on patterns in its pixels.
Object detection
A computer vision task that finds instances of chosen object types in an image or video and marks where each one appears.
Optical character recognition
A method that turns words in an image or scanned page into machine-readable text that software can search, copy and analyse.
Common misconceptions
Common misconception: Image captioning is the same as alt text.
Correction: They overlap, but they are not identical. Alt text is written for accessibility and should reflect what a reader needs in context. An automatically generated caption may describe visible items, but it may miss the purpose of the image, include irrelevant detail or fail to match accessibility guidance.
Common misconception: If the sentence sounds natural, the system probably saw the image correctly.
Correction: Fluent wording does not prove accuracy. These systems can produce convincing text while misidentifying objects, actions or relationships. The language part may hide uncertainty instead of showing it clearly.
Common misconception: It only lists objects in the picture.
Correction: Basic systems may lean in that direction, but captioning aims to describe a scene in connected language, including actions or relationships such as who is carrying what or where something is happening. Even so, it may still miss important context.
Common misconception: A captioning system understands the full situation in the image.
Correction: It matches visual patterns to likely words based on training data. It does not reliably infer unseen context such as why an event is happening, whether a person consented to being photographed, or what happened before and after the image was taken.
Quick check
Which description best matches image captioning?
Image captioning generates connected text about the scene. A single label is image classification, drawing boxes is object detection, and extracting printed text is optical character recognition.
Go deeper
Go deeper
The simple version
It looks at a picture and drafts a short description in words.
The technical version
Image captioning is a vision-language task in which a model maps an image to natural-language text. A visual component encodes features from pixels, often capturing objects, regions and relationships. A language component then predicts a sequence of words conditioned on that visual representation. Some older systems used separate vision and language modules, while many newer systems use integrated vision-language models. The output is usually evaluated with text-overlap or similarity measures, but those scores do not fully capture usefulness, factual accuracy or accessibility quality. This is why captioning should be distinguished from image classification, which predicts labels, and object detection, which localises items in the image.
Reviewed external resources
EncyclopaediaWikipedia search: Image captioning (opens in a new tab)Wikipedia
This search results page helps readers find a range of Wikipedia entries related to image captioning and suits someone who wants a broad starting point rather than one curated article.
search results, not one pagelink checked 2026-07-31
ArticleMachine Learning Glossary (opens in a new tab)Google
This glossary helps readers look up surrounding technical terms they may meet while reading about image captioning, and it suits readers who need quick definitions.
Intermediatelink checked 2026-07-31
CourseMachine Learning Crash Course (opens in a new tab)Google
This course gives foundational machine learning context that can help readers understand how systems learn patterns, and it suits beginners who want structured lessons.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Image captioning (opens in a new tab)YouTube search
This search results page offers multiple video explanations and demonstrations of image captioning, and it suits visual learners who want different teaching styles.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Image captioning (opens in a new tab)Reddit search
This search results page shows community discussions and practical questions about image captioning, and it suits readers who want informal examples and differing viewpoints.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Image captioning generates a short description from visual features. It can support search, accessibility and drafting, but it cannot guarantee that identities, relationships or context have been interpreted correctly.