Markov Chain: Become the Model
Guess the next word — and feel how a language model thinks.
Try it
Become the Model
The model is writing on its own. You're up next.
Training text (corpus)
The model learns from this text. Change it — different transitions, different feel. The model retrains instantly.
Tip: hover a candidate — you'll see how often that word occurred before. The prediction comes from exactly those occurrences.
Definition
What is a Markov chain?
Analogy
Picture your phone keyboard suggesting the next word. It has no idea what you mean to say — it just looks at the last word or two and asks: which word most often followed exactly these in the past?
A Markov chain is precisely that machine. It learned from a text that after "little" usually came "bear". So it suggests "bear" — with a certain probability. Nothing more. No understanding, just statistics about the next word.
Definition
Formally, an order-n Markov chain models a token's probability from its n predecessors only: P(wt | wt-1, …, wt-n). The "Markov assumption" says everything before that is irrelevant.
Transition probabilities are estimated by simple counting: P(next = x | context c) = count(c → x) / count(c). A language model like GPT does the same at its core — predict the next token by probability — but uses a neural network instead of a count table and thousands of context tokens instead of two or three.
In the game, a Markov chain does exactly three things:
- 1. Count transitions From your training text (the corpus), the model counts which word followed which context how often. That is the entire "training" — no neural network, just tally marks.
- 2. Propose candidates At each position it looks at the last n words and shows you the possible next words as probability bars. Tall bars = seen often, short bars = rare.
- 3. Roll the dice & continue The model draws a word according to those probabilities (sampled, not always the most frequent), appends it and asks again. Word by word, a text emerges.
The trick: drag the n slider. At n=1 the model looks at just one word — almost anything is possible, your guessing turns into gambling, the text decays into word salad. At n=3 on a small corpus there is often only one observed continuation per context — suddenly you guess nearly every word, because the model is merely quoting from memory. That is overfitting you can feel.
Check
Test your knowledge
What does a Markov chain's prediction rest on?
Select one answer
1. What does a Markov chain's prediction rest on?
2. You play on a small corpus and drag n from 1 to 3. What happens?
3. What does a high temperature do?
4. What do a Markov chain and a large language model like GPT have in common?
5. In the game the chain auto-skips some positions and doesn't ask you. Why?
Predictions from pure counting
A Markov chain describes a sequence of states in which each state depends only on the immediately preceding one. For text, this means: from a training text (the corpus) the model counts which word follows which context — and estimates from that the transition probability to the next word. No deep understanding, no grammar: just counted frequencies.
The Markov property: memoryless
The core is the Markov property: the next word depends solely on the current state — here, the last n words. Anything further back is completely ignored. The model has no long-term memory; it only knows its short window. This simplification makes the chain simple and fast, but limits how much coherence it can capture.
Where the probabilities come from
For every context, the model forms a distribution over the possible following words. A word that often followed the context in the corpus gets a high probability. The temperature shapes this distribution: a low temperature sharpens it (the favorite dominates, the text becomes predictable), a high temperature flattens it (more randomness and variety). At temperature 1, the probability exactly matches the relative frequency in the corpus.
n is the lever
The context length n decides the character of the text. At n = 1, only the last word counts: the result is grammatical word salad that sometimes sounds locally coherent but has no thread. At a large n on a small corpus, the context becomes so specific that often only one continuation exists — the model quotes the training text from memory instead of forming something new. That is overfitting, directly experienceable. In between lies the interesting range of structure and surprise.
Properties
- Memoryless: only the current state counts, not the history
- Probabilistic: the next word is rolled, not computed
- Data-driven: the model can only produce what appeared in the corpus
- Approximation: the estimated probabilities are frequencies from finite data, not true laws of language
Play with the demo! Slide n from 1 to 3 and watch word salad turn into a memorized quotation. Turn up the temperature and see the bars level out and the text grow wilder.
1906: Andrei Markov and dependent events
The Russian mathematician Andrei Markov introduced the chains named after him in 1906. He wanted to show that the law of large numbers also holds for events that depend on each other — until then, independence had been assumed for this. His model: a sequence of states in which each depends only on its direct predecessor.
1913: Vowels and consonants in Pushkin
As a practical example, Markov took on the verse novel Eugene Onegin by Pushkin. He counted by hand, across thousands of letters, how often a consonant followed a vowel and vice versa. This applied his chains to real text — one of the first statistical analyses of language of this kind.
1948: Shannon brings the chain to language
Claude Shannon picked up the idea in his groundbreaking paper A Mathematical Theory of Communication. He approximated the English language with n-gram models and generated examples that grew ever more text-like as n increased — exactly the effect this demo makes playable.
The intellectual descendants
The principle predict the next word from the context lives on today. Modern language models are vastly more powerful and not pure Markov chains, but their basic idea is the same: learn from many examples which token is most likely to come next. The simple counting chain is the ancestor of this idea.
1
# Markov chain — n-gram text generator
2
3
# 1. Training: count which word follows which context
4
function train(tokens, n):
5
table = {} # context -> {following word: count}
6
for i from 0 to len(tokens) - n - 1:
7
# context = the last n words (the state)
8
context = tokens[i .. i+n-1]
9
next_word = tokens[i+n]
10
table[context][next_word] += 1
11
return table
12
13
# 2. Prediction: distribution over the possible following words
14
function candidates(table, context, temperature):
15
counts = table[context] # only this state counts
16
# Markov property: memoryless — nothing before it counts
17
if counts is empty:
18
return []
19
# probability proportional to frequency ...
20
# ... sharpened (small) or smoothed (large) via temperature
21
for each word w in counts:
22
logit[w] = log(counts[w]) / temperature
23
P = softmax(logit) # temperature=1: P(w) = count(w) / total
24
return sort_descending(P)
25
26
# 3. Generate one word — with fallback for an unknown context
27
function next_word(tables, output, n, temperature):
28
# backoff: n -> n-1 -> ... -> 1, until a context is known
29
for k from n down to 1:
30
context = output[-k:] # the last k words
31
cand = candidates(tables[k], context, temperature)
32
if cand is not empty:
33
return sample(cand) # weighted random choice
34
return no_word
35
36
# 4. Main loop: continue the text word by word
37
function generate(corpus, n, length, temperature):
38
tokens = tokenize(corpus) # words + punctuation
39
# build one model per context length 1..n in advance
40
tables = { k: train(tokens, k) for k in 1..n }
41
output = random_starting_context(tokens, n)
42
repeat length times:
43
word = next_word(tables, output, n, temperature)
44
if no_word: break
45
output.append(word) # new state = last n words
46
return output
📊 Training: counting transitions
The generator reads through the corpus once and counts, for each context (the last n words), which word followed and how often. The result is a transition table — the model's only memory.
function train(tokens, n):
for i from 0 to len(tokens) - n - 1:
context = tokens[i .. i+n-1]
next_word = tokens[i+n]
table[context][next_word] += 1
📝 Tokenize
The corpus is broken down into words and punctuation (tokens).
📊 Count
For every context, count which word follows and how often.
📍 State
The current state is the last n words — there is no more memory than that.
📈 Distribution
From the frequencies (with temperature) comes the probability per following word.
🎲 Sample
A word is drawn with weighting, appended, the state moves forward — back to the distribution.