Probability & Expected Value
Expected value: the average of futures, weighted by probability.
A casino doesn't need luck — it needs math. With 37 roulette fields and a single green zero, every spin is an independent random event. Yet over millions of spins, the house reliably earns 2.7 cents per euro bet. How can the outcome of a single spin be unpredictable, while the long-run result is almost guaranteed?
The answer lies in three ideas: probability, distributions, and expected value. Together, they form the mathematical language that casinos, meteorologists, and AI systems use to make decisions under uncertainty.
The Language of Uncertainty
Definition
Probability
Analogy
Definition
Both interpretations produce valid numbers but answer different questions — one about repeatable experiments, the other about unique events. For a medical diagnosis, for example, you need both: how common a disease is in the population is counted the frequentist way; how certain the diagnosis is for this particular patient is estimated the Bayesian way.
Probability = relative frequency over many repetitions. Example: P(Red) = 18/37 because 18 of 37 fields are red. Works in closed systems with repeatable experiments.
Probability = degree of belief. Example: "20% chance of human-level AI by 2030" is based on prior knowledge and evidence, not counting. Works for one-off events that cannot be repeated.
Example: European Roulette
European Roulette: 37 fields
(18 red, 18 black, 1 green)
P(Red) = 18/37 ≈ 48.65%
P(Number 17) = 1/37 ≈ 2.7%
P(Red OR Black) = 36/37 ≈ 97.3%
All 37 individual probabilities
sum to exactly 1.Each field has exactly one probability, and all together they sum to 1 — that is the fundamental rule of any probability distribution. The green zero is the house edge: it shifts the odds slightly in the casino's favour. Many European casinos apply a special rule to even-money bets such as red or black: if zero comes up, the stake is not lost right away but is held for the next spin or half of it is returned. That lowers the house edge on these bets to about 1.35%. Here I calculate without that special rule.
Caution
Misconception: The Gambler's Fallacy
"Five reds in a row — black must be next!" Wrong. Each spin is independent — the wheel has no memory. P(Black) remains 18/37 on every spin, regardless of what came before. This thinking error is called the Gambler's Fallacy and stems from treating independent events as if they were dependent.
Interactive: Coin Flip Simulator
For frequentists, probability is the relative frequency over many repetitions. You can check that here: set how likely heads is and flip the coin. After a few flips, the share of heads is often far from the probability you set; after many flips, it moves closer and closer. This convergence is called the law of large numbers.
Interactive: Estimate π from pure randomness
Can you discover the circle constant π using nothing but chance? Astonishingly, yes. Throw random points into a square with a circle drawn inside it: because the probability of landing inside the circle is exactly π/4, the value 4 × hits/throws estimates π — and the more you throw, the closer you get to 3.14159. It is the same law of large numbers you saw with the coin flip above, this time turned into geometry.
The Map of All Possibilities
Definition
Distribution (PMF & PDF)
Analogy
Definition
Note
Distribution Types: An Overview
Example: Fair vs. Loaded Die
1,000 simulated rolls with a fair die: expect ~167 of each value (1-6). 1,000 rolls with a loaded die (P(6) = 0.5): expect ~500 sixes, ~100 of each other value. Each histogram sums to 1,000 rolls. The shape of the distribution — uniform or skewed — determines in ML which algorithms and preprocessing steps are appropriate.
Expected Value: The Average of the Future
The expected value E(X) is the weighted average of all possible outcomes: each outcome is multiplied by its probability and everything is summed. This number does not predict what happens on the next trial — it is the value around which the average of many trials settles. This convergence is called the Law of Large Numbers.
Note
Calculating Expected Value
Fair die:
E(X) = 1·(1/6) + 2·(1/6) + 3·(1/6) + 4·(1/6) + 5·(1/6) + 6·(1/6)
E(X) = 21/6 = 3.5
Roulette, 1 EUR on Red:
Win: +1 EUR with P = 18/37
Lose: -1 EUR with P = 19/37
E(X) = 1·(18/37) + (-1)·(19/37) = -1/37 ≈ -0.027 EUR
→ Per euro bet, the casino earns 2.7 cents.
→ Over 1 million bets: ~27,000 EUR profit.E(X) = 3.5 for the fair die — a number that never appears on a single roll, yet fairly accurately predicts the long-run average over thousands of rolls. For roulette, E(X) = -0.027 means any individual player can win in the short run, but the house edge is all but certain over time.
Note
Deal or No Deal: Expected Value vs. Gut Feeling
In the game show "Deal or No Deal," three briefcases remain: 100 EUR, 1,000 EUR, and 500,000 EUR. The expected value is (100 + 1,000 + 500,000) / 3 ≈ 167,033 EUR. The banker offers 120,000 EUR. A purely rational, risk-neutral player rejects the offer — it is below the expected value. But many real people accept, because a guaranteed 120,000 EUR feels more valuable than a 1-in-3 shot at 500,000 EUR. This gap between expected value and human behaviour was explained by Daniel Bernoulli in 1738: the subjective utility of money decreases as wealth increases.
Note
Three Steps of Understanding
- 1. Probability: Measure uncertainty as a number between 0 and 1
- 2. Distribution: Assemble all uncertainties into a complete map
- 3. Expected Value: Compress the map into a single decision-relevant number
Note
Four Layers of Understanding
From Single Event to Decision
In reinforcement learning, agents model the expected cumulative reward of each action and usually pick the action with the highest — the exploration/exploitation trade-off is fundamentally about uncertain expected values. Classifiers output probability distributions over classes; the predicted class is the one with the highest probability. Expected value and variance together quantify "how good on average" and "how much could it vary" — both are critical for real-world AI deployment. In Python, you simulate these distributions with the `random` module — for example `random.gauss()` for the normal distribution or `random.randint()` for discrete uniform.
Note
Key Takeaways
- Probability is a number between 0 and 1 that makes uncertainty measurable. Frequentists count, Bayesians estimate — both approaches are valid.
- A distribution is the complete map of all possibilities. Whether fair or "loaded" — every valid distribution sums to 1.
- Expected value compresses the map into a single decision number. It does not predict the next outcome, but it shows where the average of many trials settles — and that is exactly how casinos, insurers, and AI agents make their decisions.
Check
Quiz: Probability & Expected Value
What does a probability of 0.6 mean for an event?
Select one answer
1. What does a probability of 0.6 mean for an event?
2. A game costs 2 EUR to play. With probability 1/4 you win 10 EUR back, otherwise nothing. What is the expected value per game?
3. In roulette, red comes up five times in a row. How does the probability for black change on the next spin?
4. A researcher says: "The probability that this new drug works is 70%." Which interpretation is he using?
Check
Checkpoint: Probability & Expected Value
Can you explain this in your own words? Tick what you've got.
0 of 3 done