Time & Sequences (RNNs)

The architecture that gave AI memory — before someone found a better way.

Architectures 14 min Expert June 15, 2026

Every architecture you have encountered so far sees data as a snapshot — all at once, frozen in time. But language unfolds word by word, music note by note, and stock prices tick by tick. Rearrange the elements and the meaning changes.

Recurrent Neural Networks were the first architecture to give neural networks something resembling memory — a loop that carries information from one step to the next. This article explains how that loop works, why it inevitably fails on long sequences, and how LSTMs provided a decade-long solution before Transformers changed everything.

Why Order Matters — The Sequential Data Problem

Sequential Data & Recurrence

AnalogyDefinition
Imagine someone rips out all the pages of a detective novel and hands them to you in random order. You have all the same words and clues — but the mystery is unsolvable. The revelation on page 200 only makes sense because you read pages 1 through 199 first. A feedforward network reads the shuffled stack. An RNN reads page by page, in order.

Example

A detective novel follows a plot structure planned by an author. Real-world sequential data (e.g., stock prices) has no planned narrative — but the temporal dependency is real.

Text, time series, and audio — three domains that all share the same property: the order of elements determines meaning.

Feedforward Network

Sees all inputs simultaneously. No concept of order. Cannot distinguish 'The dog bites the man' from 'The man bites the dog.'

Recurrent Neural Network

Processes inputs step by step. The hidden state carries context from previous steps forward. Order is preserved.

'The dog bites the man' vs. 'The man bites the dog' — identical vocabulary, reversed meaning. A feedforward network treating each word independently cannot distinguish these sentences because it sees the same bag of words. An RNN processes left-to-right: by the time it reaches 'bites,' it already knows whether 'dog' or 'man' came first.

Misconception: CNNs Can Handle Sequences Too

Partial truth: 1D CNNs detect local sequential patterns effectively, but their receptive field is limited. For long-range dependencies (connecting word 1 to word 200), recurrence or attention mechanisms are necessary.

The Recurrence Loop — How RNNs Remember

Hidden State

AnalogyDefinition
Imagine a factory conveyor belt where each station (time step) receives a product and adds its own modification based on the product's current state AND what the previous station passed along (the hidden state). The work instructions (weights) are identical at every station. Moving forward along the belt, we build the product step by step — that is the forward pass. But when the quality inspector (gradient) traces a defect backward through 100 stations, the feedback signal fades at each relay — like a game of telephone where the message becomes quieter with each pass.

Example

On a conveyor belt, the physical product is altered. In an RNN, the original input remains unchanged — the hidden state is a mathematical summary, not a physical copy.

The core idea of an RNN is simple: use the same weights at every time step and carry a hidden state from step to step. This enables processing sequences of arbitrary length.

1
x₁ → h₁ First input produces first hidden state
2
x₂ + h₁ → h₂ Second input merges with memory from step 1 (the + does not mean simple addition — the network combines both through learned weights)
3
x₃ + h₂ → h₃ Third input merges with accumulated context
4
hₜ → Output Final hidden state yields the prediction
Hidden State Update Equation
hₜ = f(Wₕ · hₜ₋₁ + Wₓ · xₜ + b)

Parameter sharing: The same weight matrices W_h and W_x are reused at EVERY time step. This means: an RNN can process sequences of any length without the parameter count growing.

The Vanishing Gradient Problem

To train an RNN, the network is 'unrolled' through time and gradients are propagated backward through the entire chain — Backpropagation Through Time (BPTT). The chain rule is applied at every time step, which means multiplying by the same weight matrix. If this matrix has values consistently below 1, the gradient shrinks exponentially toward zero.

'I grew up in France. [...100 words...] I speak fluent ___.' The correct answer is 'French,' but the signal from 'France' must survive 100+ time steps. With a factor of 0.9 per step: 0.9¹⁰⁰ ≈ 0.0000265 — practically zero. The network cannot learn the connection.

Misconception: RNNs Remember Everything

No! The hidden state is a fixed-size vector. It compresses all past information into a finite representation. As new inputs arrive, older information is progressively overwritten. Even without vanishing gradients, the hidden state is a lossy summary, not a perfect recording.

Deep Dive: The Math Behind Vanishing Gradients

The gradient for the first time step is: ∂L/∂h₁ = ∂L/∂h_T · ∏(∂h_t/∂h_{t-1}) for t=2..T. Each factor contains the weight matrix W_h. If the eigenvalues of W_h are less than 1, the product converges exponentially to zero. If greater than 1, it explodes — gradient clipping caps the gradient at a threshold. LSTMs bypass this through additive updates on the cell state instead of multiplicative chaining.

Interactive: RNN Step by Step

Click through the individual time steps of an unrolled RNN. Observe how the hidden state flows from cell to cell, absorbing new input information at each step. Pay special attention to the fact that all cells share the same weights.

x₀x₁x₂x₃h₋₁h₀h₁h₂h₃RNNRNNRNNRNN
Step 1 / 7Overview: Unrolled RNN

An RNN shown unrolled through time: 4 time steps, each with its own input x. The hidden state h flows from left to right, carrying context forward.

Gates to the Rescue — LSTM and GRU

The Long Short-Term Memory (LSTM) network, invented in 1997 by Hochreiter and Schmidhuber, replaces the simple hidden state update with a memory cell controlled by three learned gates.

LSTM Cell: Information Flow Through the Gates

Forget Gate

Sigmoid: Decides what to erase from long-term memory (0 = forget, 1 = keep)

Input Gate

Sigmoid: Decides what new information to write into memory

Output Gate

Sigmoid: Decides what part of memory to expose as the current hidden state

Cell State

Long-term memory: information flows nearly unchanged (Constant Error Carousel)

Think of the LSTM cell as a secure filing cabinet with three locks. The Forget Gate lock lets you shred old documents that are no longer relevant. The Input Gate lock lets you file new important documents. The Output Gate lock controls which documents you take out to work with right now. The crucial insight: you can leave documents in the cabinet untouched for as long as they remain relevant — they do not decay over time like a vanilla RNN's hidden state.

The Constant Error Carousel — the cell state — allows gradients to flow nearly unchanged through the cell. Because the gates can learn to keep their values near 1.0, the gradient is not exponentially attenuated.

GRU: The Simplified Alternative

The Gated Recurrent Unit (GRU), introduced in 2014 by Cho et al., simplifies the LSTM architecture to two gates (Update and Reset) and achieves comparable performance with fewer parameters.

1986

RNN Concept

Rumelhart et al. describe the idea of recurrent connections in neural networks

1991

Vanishing Gradient Identified

Hochreiter shows in his diploma thesis that gradients vanish exponentially in deep recurrent networks

1997

LSTM

Hochreiter & Schmidhuber invent Long Short-Term Memory — gates control information flow

2014

GRU

Cho et al. simplify LSTM to two gates — comparable performance with fewer parameters

2016

Google NMT

Google Translate switches to LSTM-based Neural Machine Translation — dramatic quality improvement

2017

Transformers

Vaswani et al. replace recurrence with Self-Attention — parallel processing of all positions

Real-World Example: Google Translate

Before 2016, Google Translate used phrase-based statistical models that translated sentence fragments independently — often producing grammatically incorrect output. In 2016, Google switched to Neural Machine Translation powered by deep LSTMs, producing dramatically more fluent translations. In 2017, Transformers replaced LSTMs, adding another leap — foreshadowing the next article in your learning path.

Misconception: LSTMs Have Infinite Memory

No! LSTMs forget too — but in a controlled, learned way rather than the uncontrolled exponential decay of vanilla RNNs. The Forget Gate actively decides what to discard. Over very long sequences, even LSTMs struggle with very distant dependencies.

Despite their success, both architectures retain the fundamental sequential processing constraint: each step must wait for the previous step to complete, making parallelization on GPUs impossible. This sequential bottleneck directly motivated the Transformer architecture — the next article in your learning path.

Deep Dive: The RNN Renaissance

Transformers have quadratic cost with sequence length (O(n²) attention). State Space Models like Mamba (2023) achieve linear scaling through a learned recurrence. Streaming applications (real-time audio, sensor monitoring) where latency is critical benefit from RNN-like architectures. Hybrid architectures combine attention and recurrence. The key message: the recurrence principle is not dead — it has evolved.

Key Takeaways

  • Sequential data carries meaning in order — feedforward networks destroy this by processing everything simultaneously, while RNNs process step-by-step with a hidden state that accumulates context.
  • The Vanishing Gradient Problem is not a bug but a mathematical inevitability: repeated multiplication through long chains causes gradients to shrink exponentially, erasing long-range memory.
  • LSTMs solve this with gates that learn WHAT to remember and forget — but their strictly sequential processing makes them slow, setting the stage for Transformers.

Check Your Understanding

  • Why can't feedforward networks distinguish 'The dog bites the man' from 'The man bites the dog'?
  • A gradient factor of 0.85 applied 150 times — what happens to the signal?
  • Which gate of an LSTM cell would you use to erase irrelevant information from the cell state?

Quiz: Recurrent Neural Networks

Question 1 / 6
Not completed

What is the primary purpose of the hidden state in a Recurrent Neural Network?

Select one answer
Answer Key: 1) B · 2) C · 3) B · 4) B · 5) B · 6) C