Back to Learning Area

Probability & Expected Value

Expected value: the average of futures, weighted by probability.

  • Fundamentals
  • 14 min
  • Beginner

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Understanding probability

What is probability?

Probability is a number. It shows: how often does something happen? A coin shows heads in half of the flips. The probability for that is 50 percent. All probabilities together always result in 100 percent.

Why casinos always win

The casino needs no luck. It calculates with probabilities. In roulette, the casino always has a small advantage. With many games, it earns reliably. This is called expected value. Expected value means: what do you win on average?

Probability and AI

AI systems use probabilities. An example is language models. Language models are AI programs for text. They calculate: which word comes next? They choose the most likely word. This is how naturally sounding sentences emerge.

A casino doesn't need luck — it needs math. With 37 roulette fields and a single green zero, every spin is an independent random event. Yet over millions of spins, the house reliably earns 2.7 cents per euro bet. How can the outcome of a single spin be unpredictable, while the long-run result is almost guaranteed?

The answer lies in three ideas: probability, distributions, and expected value. Together, they form the mathematical language that casinos, meteorologists, and AI systems use to make decisions under uncertainty.

The Language of Uncertainty

Definition

Probability

Analogy

A roulette wheel is a closed system with exactly 37 possible outcomes and known, fixed probabilities — P(Red) = 18/37 ≈ 48.65%. The question "What is the probability that AI reaches human-level intelligence by 2030?" is a one-time event — you cannot repeat it 1,000 times and count. Here, an expert estimates the probability based on prior knowledge and new evidence: a degree of belief, not a relative frequency.

Both interpretations produce valid numbers but answer different questions — one about repeatable experiments, the other about unique events. For a medical diagnosis, for example, you need both: how common a disease is in the population is counted the frequentist way; how certain the diagnosis is for this particular patient is estimated the Bayesian way.

Frequentist (Counting)

Probability = relative frequency over many repetitions. Example: P(Red) = 18/37 because 18 of 37 fields are red. Works in closed systems with repeatable experiments.

Bayesian (Believing)

Probability = degree of belief. Example: "20% chance of human-level AI by 2030" is based on prior knowledge and evidence, not counting. Works for one-off events that cannot be repeated.

Example: European Roulette

European Roulette: 37 fields
(18 red, 18 black, 1 green)

P(Red)           = 18/37 ≈ 48.65%
P(Number 17)     = 1/37  ≈ 2.7%
P(Red OR Black)  = 36/37 ≈ 97.3%

All 37 individual probabilities
sum to exactly 1.

Each field has exactly one probability, and all together they sum to 1 — that is the fundamental rule of any probability distribution. The green zero is the house edge: it shifts the odds slightly in the casino's favour. Many European casinos apply a special rule to even-money bets such as red or black: if zero comes up, the stake is not lost right away but is held for the next spin or half of it is returned. That lowers the house edge on these bets to about 1.35%. Here I calculate without that special rule.

Caution

Misconception: The Gambler's Fallacy

"Five reds in a row — black must be next!" Wrong. Each spin is independent — the wheel has no memory. P(Black) remains 18/37 on every spin, regardless of what came before. This thinking error is called the Gambler's Fallacy and stems from treating independent events as if they were dependent.

Interactive: Coin Flip Simulator

For frequentists, probability is the relative frequency over many repetitions. You can check that here: set how likely heads is and flip the coin. After a few flips, the share of heads is often far from the probability you set; after many flips, it moves closer and closer. This convergence is called the law of large numbers.

Interactive: Estimate π from pure randomness

Can you discover the circle constant π using nothing but chance? Astonishingly, yes. Throw random points into a square with a circle drawn inside it: because the probability of landing inside the circle is exactly π/4, the value 4 × hits/throws estimates π — and the more you throw, the closer you get to 3.14159. It is the same law of large numbers you saw with the coin flip above, this time turned into geometry.

The Map of All Possibilities

Definition

Distribution (PMF & PDF)

Analogy

A fair die has six equal bars in its distribution diagram — each face appears 1/6 of the time. A loaded die with P(6) = 0.5 and P(1-5) = 0.1 each has one tall bar and five short ones. Both are valid distributions because all values sum to 1. "Loaded" does not mean "broken" — it simply reflects a different reality.

Note

Distribution Types: An Overview

Uniform Distribution All outcomes equally likely. Example: fair die (1/6 each).
Normal Distribution Bell-shaped, symmetric around the mean. Example: heights, measurement errors.
Poisson Distribution Counts rare events in fixed time intervals. Example: customer calls per hour.
Binomial Distribution Number of successes in n yes/no trials. Example: number of heads in 10 coin flips.

Example: Fair vs. Loaded Die

1,000 simulated rolls with a fair die: expect ~167 of each value (1-6). 1,000 rolls with a loaded die (P(6) = 0.5): expect ~500 sixes, ~100 of each other value. Each histogram sums to 1,000 rolls. The shape of the distribution — uniform or skewed — determines in ML which algorithms and preprocessing steps are appropriate.

Expected Value: The Average of the Future

The expected value E(X) is the weighted average of all possible outcomes: each outcome is multiplied by its probability and everything is summed. This number does not predict what happens on the next trial — it is the value around which the average of many trials settles. This convergence is called the Law of Large Numbers.

Note

Calculating Expected Value

Fair die:
E(X) = 1·(1/6) + 2·(1/6) + 3·(1/6) + 4·(1/6) + 5·(1/6) + 6·(1/6)
E(X) = 21/6 = 3.5

Roulette, 1 EUR on Red:
Win: +1 EUR with P = 18/37
Lose: -1 EUR with P = 19/37
E(X) = 1·(18/37) + (-1)·(19/37) = -1/37 ≈ -0.027 EUR

→ Per euro bet, the casino earns 2.7 cents.
→ Over 1 million bets: ~27,000 EUR profit.

E(X) = 3.5 for the fair die — a number that never appears on a single roll, yet fairly accurately predicts the long-run average over thousands of rolls. For roulette, E(X) = -0.027 means any individual player can win in the short run, but the house edge is all but certain over time.

Note

Deal or No Deal: Expected Value vs. Gut Feeling

In the game show "Deal or No Deal," three briefcases remain: 100 EUR, 1,000 EUR, and 500,000 EUR. The expected value is (100 + 1,000 + 500,000) / 3 ≈ 167,033 EUR. The banker offers 120,000 EUR. A purely rational, risk-neutral player rejects the offer — it is below the expected value. But many real people accept, because a guaranteed 120,000 EUR feels more valuable than a 1-in-3 shot at 500,000 EUR. This gap between expected value and human behaviour was explained by Daniel Bernoulli in 1738: the subjective utility of money decreases as wealth increases.

Note

Three Steps of Understanding

  1. 1. Probability: Measure uncertainty as a number between 0 and 1
  2. 2. Distribution: Assemble all uncertainties into a complete map
  3. 3. Expected Value: Compress the map into a single decision-relevant number

Note

Four Layers of Understanding

From Single Event to Decision

Single Event: "What can happen?"
Distribution: "How likely is each outcome?"
Expected Value: "What is the average result?"
Decision: "What should I do?"

In reinforcement learning, agents model the expected cumulative reward of each action and usually pick the action with the highest — the exploration/exploitation trade-off is fundamentally about uncertain expected values. Classifiers output probability distributions over classes; the predicted class is the one with the highest probability. Expected value and variance together quantify "how good on average" and "how much could it vary" — both are critical for real-world AI deployment. In Python, you simulate these distributions with the `random` module — for example `random.gauss()` for the normal distribution or `random.randint()` for discrete uniform.

Note

Key Takeaways

  1. Probability is a number between 0 and 1 that makes uncertainty measurable. Frequentists count, Bayesians estimate — both approaches are valid.
  2. A distribution is the complete map of all possibilities. Whether fair or "loaded" — every valid distribution sums to 1.
  3. Expected value compresses the map into a single decision number. It does not predict the next outcome, but it shows where the average of many trials settles — and that is exactly how casinos, insurers, and AI agents make their decisions.

Check

Quiz: Probability & Expected Value

Not completed

What does a probability of 0.6 mean for an event?

Select one answer

Answer Key: 1) B · 2) B · 3) C · 4) B

Check

Checkpoint: Probability & Expected Value

Can you explain this in your own words? Tick what you've got.

0 of 3 done