Learn AI /Neural Networks /Recurrent neural network
Recurrent neural networkRNN
- Deeper
- Neural Networks
- Technical readers
Also called: RNN, LSTM.
Back to the milestone you came from
In five seconds
What it is
A recurrent neural network processes a sequence by carrying a changing hidden state from one step to the next, so earlier inputs can influence later outputs.
In plain words
An RNN reads a sequence one piece at a time and carries forward a running summary so what came earlier can affect what comes next.
An everyday example
Someone gives you directions one turn at a time. After each instruction, you update a small sketch of where you think you are. You interpret the next turn using both the new instruction and the sketch carried forward, rather than starting again from a blank page. The sketch is like the hidden state, and updating it by the same rule after every instruction resembles the recurrent operation. On a very long route, early details may fade from the sketch, just as a simple recurrent neural network can struggle to preserve information across many steps.
A librarian consciously chooses what to write down, while an RNN does not have awareness or explicit reasoning. Its hidden state is just numbers updated by learned rules. Also, a real librarian can jump back, ask clarifying questions, or search many shelves at once; a standard RNN usually processes in sequence and can lose older information unless its design, such as an LSTM, helps preserve it.
Remember thisAn RNN processes ordered inputs by applying the same learned operation at each step and carrying forward a changing hidden state that summarises relevant earlier inputs.
Understand it
Understand it
Why it matters
If you read an AI proposal, audit note, model card, or vendor presentation, you may see RNN or LSTM used for speech, translation, forecasting, or anomaly detection. Knowing the term helps you ask practical questions: Does the model process data in order? How well can it handle long sequences? Is it producing one result at the end or many along the way? Could a transformer or another method be more suitable? This matters for procurement, risk assessment, compute cost, latency, and performance claims. It also helps you recognise that older sequence models may still be appropriate for small devices or short time-series, even if they are no longer the main choice for many language systems.
How an RNN carries information through a sequence
After viewing this, a reader should be able to say that an RNN reuses the same cell at each step and passes a hidden state forward so earlier inputs can affect later outputs.
01
Step 1: First item enters
The first cell takes the first input item and an initial hidden state, often zeros, then creates a new hidden state.
Draw cell 1 as a rounded rectangle labelled 'shared RNN cell'. Place x1 below it, h0 on the left, h1 on the right, and optional y1 above. Colour hidden-state arrows blue and input arrows grey.
02
Step 2: Same cell logic repeats
The second step uses the same learned weights as the first step. Only the input and hidden-state values change.
Draw cell 2 identical to cell 1 with labels x2, h1, h2, and optional y2. Add a small note above both cells: 'same weights reused at each step'.
03
Step 3: Information is carried forward
The hidden state acts as a running summary. It lets earlier inputs influence later computation.
Extend the chain to cells 3 and 4 with x3, x4 and h2, h3, h4. Make the blue hidden-state arrows visually continuous from left to right.
04
Step 4: Outputs can be per step or final
Some tasks use every output, while others use only the last output or last hidden state.
Above each cell, show faint y1 to y4. Add two braces: one over all y values labelled 'output each step' and one around y4 labelled 'or final output only'.
Show a left-to-right chain of identical RNN cells. Each cell receives the current input from below and the previous hidden state from the left, then emits a new hidden state to the right and an output above. Read the diagram step by step from x1 to x4. Use matching colours for all hidden-state arrows to show continuity across time.
Hovering or tapping a cell highlights the current input, previous hidden state, new hidden state, and optional output, while a toggle switches between 'many-to-many' and 'many-to-one' output use.
Describe this without the diagram
A four-step horizontal diagram shows identical RNN cells repeated across time. At the first step, input x1 and an initial hidden state h0 enter the first cell. The cell outputs a new hidden state h1 and an optional output y1. The next cell receives x2 and h1, producing h2 and y2, and this repeats through x4. A note says the same weights are reused in every cell. The key point is that the hidden state moves forward across the sequence, carrying a numerical summary of earlier inputs. Another note shows that some tasks use outputs at every step, while others use only the final output.
With reduced motion: Show the same four-cell chain as a static diagram with all arrows visible at once. Use colour and labels, not animation, to indicate that the hidden state passes from one step to the next and that the same cell logic is reused.
Many AI tasks involve sequences: words in a sentence, notes in speech, sensor readings over time, or clicks in order. A recurrent neural network handles this by reading one item at a time instead of treating every item as unrelated.
At each step, it takes two things as input: the current item and a hidden state. The hidden state is a vector, meaning a list of numbers, that acts like a running summary of what came before. The network updates that summary using learned weights, which are numbers that control how strongly signals affect later calculations.
Because the same update rule is reused at every step, the model can process sequences of different lengths. It can produce one output at the end, such as a sentiment label for a whole sentence, or an output at each step, such as the next word or a tag for each token.
In practice, basic RNNs often struggle with long-range dependencies, meaning information from far back in the sequence can fade or become unstable during training. Variants such as LSTM and GRU were designed to keep useful information for longer. RNNs are important historically and still useful in some time-series and compact systems, but many language tasks now use transformers instead.
How “Updating a sketch as directions arrive” maps onto the real thing
| In the story | In the AI system |
|---|---|
| each clue from the visitor | each item in the input sequence |
| the librarian's notepad | the hidden state carried from one step to the next |
| updating the note after every clue | recurrent state update using shared weights |
| the final shelf choice | the output prediction at the end or at a step |
A worked example
Forecasting the next status in a sequence of monthly workplan updates.
Starting from: Illustrative sequence: "on schedule", "minor delay", "inputs pending"
- The model starts with an initial hidden state and reads "on schedule", producing a new state that summarises the first update.
- It then reads "minor delay" and combines that input with the previous hidden state. The new state therefore reflects both reporting periods.
- Next, it reads "inputs pending" and updates the state again. The same learned weights are used at each step, although the hidden state changes.
- An output layer uses the final hidden state to score possible next statuses. During training, errors are propagated backwards through the unfolded sequence using backpropagation through time.
Result: In this illustrative run, the highest-scoring next status is "at risk of delay". That output is influenced by the order of all three updates, not only by the final phrase.
In a United Nations workplace
A chief of section in a technical division is clearing a note for the file, due to the front office by close of business, in response to an audit observation on a model that forecasts the section's document-processing backlog. The vendor describes it as recurrent, but the file does not explain what information passes from one reporting period to the next.
The chief needs to confirm whether the model processes the figures in sequence, updates a hidden state at each step and uses shared parameters across the sequence. That helps distinguish an RNN from a model that treats each reporting period independently, and exposes questions about long-range dependencies that the vendor summary has politely avoided.
Before you try thisDo not upload audit working papers, underlying case records or personal or commercially sensitive data to a tool that has not been approved for that material. A model's recurrent architecture also says nothing by itself about forecast quality, appropriate controls or fitness for the operational decision.
Where you may hear it
- A vendor demo during a procurement meeting on forecasting tools
- An internal audit exit meeting on controls around a sequence model
- A technical review of a monitoring dashboard before clearance
- A briefing to an ICT governance board on model architecture
Avoid confusion
Avoid confusion
Often confused with
Transformer
A neural network design that processes many parts of a sequence in parallel by weighting which parts matter most to each other.
Convolutional neural network
A layered model that reuses small filters across a grid-like input, combines local patterns into larger features, and produces task outputs from them.
Brain analogy (and its limits)
A comparison between neural networks and brains that can help with intuition, but breaks down because AI systems use simplified maths rather than living cells and biological processes.
Common misconceptions
Common misconception: An RNN stores the whole past exactly like a recording.
Correction: It does not keep a perfect copy of every earlier input. It compresses prior context into a hidden state, which is a limited numerical summary. That summary can preserve useful patterns, but some detail may fade, especially in basic RNNs over long sequences.
Common misconception: LSTM and RNN are completely different model families.
Correction: An LSTM is a specific kind of recurrent neural network. People often contrast 'RNNs' with 'LSTMs', but in a broad sense LSTMs are designed to solve weaknesses of simpler recurrent units by controlling what information is kept, updated, or forgotten.
Common misconception: Because it reads in order, an RNN automatically understands sequence better than other models.
Correction: Reading in order can help for some tasks, but it does not guarantee better results. Basic RNNs can struggle with long-range dependencies and slow training. Transformers often outperform them on long text because they can relate distant parts of the sequence more directly.
Quick check
What makes a recurrent neural network different from a basic feed-forward network?
The defining feature is recurrence: the same learned update is applied across sequence steps while a hidden state passes forward. The other choices wrongly claim that weights change at every step, input and output lengths must match, or all positions are combined in parallel.
Go deeper
Go deeper
The simple version
An RNN reads a sequence one piece at a time and carries forward a running summary so what came earlier can affect what comes next.
The technical version
At time step t, a recurrent model computes a new hidden state from the current input xt and the previous hidden state h(t-1), typically with shared parameters across all time steps. In a simple RNN, this may look like h(t) = f(Wx xt + Wh h(t-1) + b), where f is an activation function. Outputs may be produced from each h(t) or only from the final state. Training usually uses backpropagation through time, which unfolds the repeated cell across steps and propagates gradients through that chain. This creates well-known vanishing and exploding gradient problems, which is why gated variants such as LSTMs and GRUs were introduced. Compared with convolutional neural networks, which scan local patterns across grid-like inputs, recurrent networks are built around ordered dependence across time or position. Compared with transformers, RNNs pass information step by step through a hidden state rather than weighting relationships among many positions in parallel.
Reviewed external resources
EncyclopaediaRecurrent neural network (opens in a new tab)Wikipedia
This link gives a broad reference-style overview of recurrent neural networks and suits readers who want definitions, variants, history, and terminology in one place.
Intermediatelink checked 2026-07-31
InteractiveTensorFlow Playground (opens in a new tab)TensorFlow
This link gives a hands-on interactive way to explore neural network behaviour and suits readers who learn best by adjusting settings, even though it is not specific to recurrent networks.
Starterlink checked 2026-07-31
VideoNeural networks, explained visually (opens in a new tab)3Blue1Brown
This link gives an intuitive visual explanation of neural networks and suits readers who want stronger foundations before tackling recurrent models in more detail.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Recurrent neural network (opens in a new tab)YouTube search
This link is a search results page for many video explanations of recurrent neural networks and suits readers who want to compare teaching styles and depth levels.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Recurrent neural network (opens in a new tab)Reddit search
This link is a search results page for community discussions about recurrent neural networks and suits readers who want practical questions, opinions, and troubleshooting threads.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
An RNN processes ordered inputs by applying the same learned operation at each step and carrying forward a changing hidden state that summarises relevant earlier inputs.