Learn AI /AI Agents and Automation /Short-term memory (in AI systems)

Short-term memory (in AI systems)

Also called: working memory, session memory.

Detail

In five seconds

What it is

Temporary information kept available during a current interaction so an AI system can use recent instructions, results and observations in its next step.

In plain words

It is the temporary notepad an AI system uses during one conversation or task so it can keep track of the latest instructions and results.

An everyday example

A warehouse clerk in Panama City is checking whether a delivery matches the shipping manifest. On a small table, the clerk keeps only the papers needed for this one truck: the manifest, a note about two damaged boxes, the pallet count, and the latest message from the driver. As each pallet is unloaded, the clerk updates the note, crosses off completed checks, and keeps the most relevant papers visible. When the truck leaves, those working papers are cleared from the table unless someone files them elsewhere.

Where the story stops working

A clerk can deliberately understand the meaning of each paper and choose what matters in a rich human way. An AI system does not have that kind of human understanding; it processes text and structured data according to its design. Also, a real table has a fixed visible space, but AI short-term memory may be rebuilt, summarised, or selectively retrieved in ways the table image does not show.

Remember thisShort-term memory is temporary working state for the current interaction, often stored outside the model and supplied again on later calls. It supports continuity, but neither guarantees accuracy nor implies immediate deletion.

Understand it

Why it matters

For non-specialists, this term matters because it affects whether an AI system stays consistent during a task. Good short-term memory can help a system remember the latest instruction, use a tool result correctly, and avoid repeating or skipping steps. Weak short-term memory can lead to contradictions, lost constraints, duplicate actions or answers that ignore recent corrections. If you are choosing, supervising or auditing an AI system, this helps you ask practical questions: What does it keep during a session? For how long? Can it be updated, summarised or cleared? Does a human review important steps? Those questions matter for quality, safety and accountability.

How short-term memory supports the next step

Show how temporary session information is collected, updated and reused to guide the next action in an AI system.

01
1. New input arrives
The system receives a user message or a new event in a task.
Draw a message card entering the first box labelled 'new input'. Add example text: 'Check whether 120 cartons match the manifest'.
02
2. Temporary state is assembled
The system gathers the recent conversation, current task notes, constraints and any useful recent observations for this session.
Second box labelled 'short-term memory'. Show stacked cards: 'recent messages', 'current plan', 'tool results', 'rules for this task'.
03
3. Next step is chosen
Using that temporary state, the system prepares the next model call or action, such as asking a question or calling a tool.
Third box labelled 'next step'. Split icon options: speech bubble for a question, wrench for a tool call.
04
4. Results update the state
If a tool returns data, or the user answers, the temporary state is updated so the task can continue coherently.
Fourth box labelled 'update memory'. Show an arrow from a small tool result card into the memory stack; one old note fades or is summarised.
05
5. Output for this turn
The system produces a reply or action based on the updated temporary state. When the session ends, this working state may be discarded unless separately saved.
Fifth box labelled 'reply or action'. Add a small footer note: 'may end here or continue to another turn'.

A left-to-right step diagram with five boxes. Read from the user request to temporary memory, then to a tool call, then a memory update, then the next reply or action. Small side labels show examples of what is kept, such as recent instructions, tool results and pending tasks.

On hover or tap, each example card reveals a short sample such as a user instruction, a tool response, or a pending subtask. A replay button highlights the cycle from step 2 to step 4 for multi-step tasks.

Describe this without the diagram

The diagram explains short-term memory as a five-step process. First, a new message or event arrives. Second, the system gathers temporary session information such as recent messages, current task notes, constraints and recent tool results. Third, that temporary information is used to choose the next step, such as replying or calling a tool. Fourth, any new result updates the temporary information, sometimes replacing or summarising older details. Fifth, the system produces the current reply or action. The key point is that this memory is actively updated during a task and may be discarded when the task ends unless saved elsewhere.

With reduced motion: A static five-box diagram shows the same sequence without animation: new input, short-term memory, next step, update memory, reply or action. Dashed arrows indicate that after an update, the system can loop back and continue the task.

Short-term memory in an AI system is the temporary working information kept during one ongoing task or session. It can include the user’s recent instructions, earlier messages, tool results, notes created by the system, and the current plan. This helps the system continue from one step to the next instead of starting fresh each time.

What goes in depends on the system design. Inputs can include the latest user message, selected parts of the conversation so far, outputs from tools such as a calendar or database, and reminders like “use British spelling” or “check totals before replying”. A tool is a connected external service or function the system can call.

What happens next is usually simple but important: the system stores, updates, trims, or rewrites the temporary information so the next model call has the right context. “Context” here means the information provided to the model for the current step. Some systems keep a running scratchpad or state object; others rebuild the temporary memory each turn from recent events.

What comes out is not the memory itself but a better next action or reply. For example, the system may ask a clearer follow-up question, call the right tool, avoid repeating a step, or carry forward a constraint from earlier in the session. This is different from long-term memory, which is meant to persist across sessions, and different from the context window, which is the model’s size limit for how much information can be included at once.

How “Checking a shipping manifest at a loading bay” maps onto the real thing

In the story	In the AI system
Papers kept on the table for one truck	Temporary information kept for one task or session
Latest driver message and damage note	Recent user instructions and fresh observations
Crossing off checks as pallets are verified	Updating working state after each step or tool result
Clearing the table after the truck leaves	Short-term memory ending when the session or task ends

A worked example

A statistician is checking three indicators in the procurement unit’s draft workplan while incorporating comments from the annual review.

Starting from: “Check the indicator table. Keep baselines separate from annual targets, flag missing values and use the wording already cleared by the chief of section.”

The application creates session state containing the drafting instruction, the three indicator rows, the clearance constraint and a pending task list.
A spreadsheet tool flags one missing baseline and one target entered in the wrong column. Its output is added to the session state.
On the next model call, the application supplies a summary of that state. The agent asks whether the missing baseline should remain blank or be marked for follow-up.
The statistician replies, “Leave it blank and add it to the follow-up list.” The state is updated, and the next call receives that decision together with the earlier formatting and clearance instructions.

Result: The revised table preserves the cleared wording, separates baselines from targets and carries the unresolved baseline into the follow-up list. This continuity comes from temporary session state passed across calls, not from a permanent change to the model.

In a United Nations workplace

At the procurement unit’s annual workplan review, a statistician from the data unit uses an approved internal agent to check the draft results framework before close of business. Across several turns, the agent keeps the current indicators, the chief of section’s formatting instruction, a spreadsheet check and unresolved questions available for the next step.

The model does not necessarily retain these details itself. The application may hold them in session state, then inject or summarise the relevant instructions, tool outputs and pending tasks for each successive model call. That is how the agent can continue the same review without being briefed again at every turn.

Before you try thisBids, supplier evaluations, pricing details, personal data and other procurement-sensitive material must not be entered into a tool unless it is specifically approved for that material. Session-scoped memory does not guarantee deletion: prompts and tool outputs may still be logged or retained. The statistician must also verify every proposed correction against the file.

Where you may hear it

During a vendor demonstration to an ICT governance board
At an annual workplan review when an agent forgets an earlier instruction
In a design review for a multi-step automation
During a procurement unit’s technical evaluation of an internal assistant

Avoid confusion

Often confused with

Side-by-side

Context window vs memory
If it has a million-token context window, does it remember me?

Common misconceptions

Common misconception: Short-term memory means the model permanently remembers me or my organisation.
Correction: Usually it refers to temporary information kept for the current task or session. It may be cleared when the session ends unless the system has a separate long-term memory feature or another storage system. Whether anything persists depends on the product design, not the phrase alone.
Common misconception: Short-term memory and context window are the same thing.
Correction: They are related but not identical. The context window is the model’s capacity limit for the information provided at one time. Short-term memory is the working information a system tries to keep available during a task. A system may manage that memory by selecting, summarising or rotating what fits into the context window.
Common misconception: If a system has short-term memory, it will reliably follow every earlier instruction.
Correction: Not necessarily. The temporary memory may be incomplete, badly selected, overwritten by later instructions, or too large to include fully in the next model call. The system can still miss details or use them poorly, so testing and human oversight are still needed.

Quick check

Which statement best describes short-term memory in an AI system?

Go deeper

The simple version

It is the temporary notepad an AI system uses during one conversation or task so it can keep track of the latest instructions and results.

The technical version

In many AI agents, short-term memory is a session-scoped working state used across successive model calls. It may contain recent dialogue turns, system instructions, intermediate reasoning notes if the design exposes them, tool outputs, pending subtasks, execution state and user-specific constraints for the current task. This state is often stored outside the model itself, then injected, summarised or transformed before each call. The mechanism can be as simple as passing the latest messages, or more structured, such as maintaining a state object with fields for observations, actions taken, plan status and tool responses. It is distinct from long-term memory because it is not primarily designed for persistence across sessions, and distinct from the context window because the working state may be larger than what is sent at once and therefore needs selection or compression.

Reviewed external resources

Primary sourceModel Context Protocol (opens in a new tab)Model Context Protocol
This official Model Context Protocol material helps readers understand how tools and context can be connected in structured ways, and suits technical readers.
Deeperlink checked 2026-07-31
EncyclopaediaShort-term memory (opens in a new tab)Wikipedia
This Wikipedia page gives a broad introductory overview of the term and suits readers who want a quick general orientation.
Intermediatelink checked 2026-07-31
ArticleBuilding effective agents (opens in a new tab)Anthropic
This Anthropic article explains agents in practical terms and suits readers who want to see how temporary working state fits into agent behaviour.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Short-term memory (in AI systems) (opens in a new tab)YouTube search
This YouTube search results page helps readers find different video explanations of the term and suits people who prefer audiovisual learning; it is a search page, not a single lesson.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Short-term memory (in AI systems) (opens in a new tab)Reddit search
This Reddit search results page helps readers see community questions and informal explanations, and suits readers who want plain-language discussion; it is a search page, not one authoritative source.
search results, not one pagelink checked 2026-07-31

Where this came from

Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.

The one thing to remember

Short-term memory is temporary working state for the current interaction, often stored outside the model and supplied again on later calls. It supports continuity, but neither guarantees accuracy nor implies immediate deletion.