Back to Learning Area

Bayes & Conditional Probability

Conditional probability: the tool that identifies statisticians — they do the math instead of guessing.

  • Fundamentals
  • 15 min
  • Beginner

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Why a test can trick you

What this page explains

This page explains conditional probability. This means: new information changes the probability. A test is 95 percent accurate. The result is positive. Still, only 16 percent of positive results are really sick.

The Bayes theorem

The Bayes theorem is a formula. It calculates new probabilities. You start with an initial value. This is called a prior. Then you see new clues. The formula calculates the new value. The new value is called a posterior.

How AI uses this

AI systems use Bayes to learn. A spam filter is a program. It sorts out unwanted emails. The word win makes an email suspicious. The filter calculates the spam probability. Every word gives a new clue.

A medical test is 95% accurate. The disease affects 1% of people. You test positive. How likely are you to actually be sick? The most common answer: "95%." The real answer: about 16%. This is not a trick — it is Bayes' theorem in action.

Your gut feeling is often way off with probabilities — especially when rare events are involved. This article walks you through three tools: conditional probability (what "given that" means mathematically), Bayes' theorem (the formula for reversing probabilities), and Bayesian updating (how AI systems learn from data).

Conditional Probability — "Given That..."

Definition

Conditional Probability

Analogy

You draw a card from a standard 52-card deck. P(Ace of Hearts) = 1/52. Someone tells you: "The card is red." This eliminates 26 cards. Your new universe is 26 red cards. P(Ace of Hearts | red) = 1/26. The information "red" halved your possibilities and doubled your probability. That is conditional probability: new information shrinks the possibility space.

Example

The card analogy works perfectly because the numbers are exact and small enough to verify by hand. In reality, you work with estimated frequencies (disease prevalence, test accuracy), not perfectly known card counts.

Example: The Medical Test

1% of the population has disease X. The test detects sick people 95% of the time (sensitivity). In healthy people, it falsely shows positive 5% of the time. Take 1,000 people:

  1. 1. 1,000 people: 10 sick, 990 healthy (completed)
  2. 2. Of 10 sick: 9 test positive95% would be 9.5 (completed)
  3. 3. Of 990 healthy: 50 false positives5% would be 49.5 (completed)
  4. 4. Total positive: 9 + 50 = 59 (completed)
  5. 5. P(sick | positive) = 9/59 ≈ 15%without rounding to whole people: 16% (current step)

You test positive. The probability of actually being sick: only 16%. The 990 healthy people produce 50 false alarms that flood the 9 true positives. The base rate (1% prevalence) dominates.

Caution

The Base Rate Fallacy

Do not confuse P(positive | sick) with P(sick | positive). Test accuracy (how well the test detects the sick) and diagnostic probability (how likely you are sick given a positive test) are completely different things. The base rate — how rare the disease is — makes all the difference.

Caution

Misconception: 95% Accurate Test = 95% Probability of Being Sick

95% describes P(positive | sick) — how well the test finds sick people. But you want P(sick | positive) — how likely you are sick. Without the base rate (how common the disease is), the accuracy number is meaningless. For rare diseases, even excellent tests produce mostly false alarms.

Bayes' Theorem — The Formula for Updating Beliefs

Definition

Bayes' Theorem

Analogy

You hear a noise at night. Your initial beliefs (priors): P(cat knocked something over) = 80%, P(burglar) = 0.1%. Then you hear glass breaking — P(glass | cat) = 10%, but P(glass | burglar) = 90%. Bayes updates your belief: P(burglar) rises significantly because glass breaking is much more likely under the burglar scenario. But the extremely low prior pulls the posterior back — it does not jump to 90% but (if only the cat or a burglar can explain the noise) to just over 1%. The prior and the likelihood compete, and Bayes mediates.

Example

Humans do not compute numbers at night — they react with gut feelings and heuristics. Bayes requires explicit probabilities. This gap between intuitive and mathematical updating is exactly what the article teaches.

The Four Building Blocks

Prior P(A) Your belief about A before seeing evidence B (e.g., disease is rare: 1%)
Likelihood P(B|A) How probable the evidence B is if A is true (e.g., test positive if sick: 95%)
Evidence P(B) The total probability of seeing B across all scenarios
Posterior P(A|B) Your updated belief about A after seeing B — the answer you actually want

Worked Example

P(sick | positive) = P(positive | sick) x P(sick) / P(positive)
                    = 0.95 x 0.01 / 0.059
                    = 0.0095 / 0.059
                    ≈ 0.161 → about 16.1%

P(positive) = P(positive | sick) x P(sick) + P(positive | healthy) x P(healthy)
            = 0.95 x 0.01 + 0.05 x 0.99
            = 0.0095 + 0.0495
            = 0.059
Frequency Table

Count 1,000 people, divide positive-tested sick by all positives. Intuitive, but only practical with small numbers.

Bayes' Formula

Prior x Likelihood / Evidence. Same answer (16%; the table shows 15% only because it rounds to whole people), but universally applicable — even when you cannot count 1,000 people.

Caution

Misconception: Strong Evidence Overrides Everything

Even if P(B|A) is high — the likelihood alone does not determine the result. With an extremely low prior (e.g., a disease affects 0.01%), the posterior stays low even with a 99% test. The prior always pulls the result in its direction. Only with moderate priors or multiple independent pieces of evidence can the evidence overcome the prior.

Interactive: Compute Bayes' Theorem

Adjust the prior, sensitivity, and false positive rate and watch how the posterior changes. Try the predefined scenarios from the article (rare disease, spam filter) and experiment with your own values.

Bayesian Updating — Learning Step by Step

Definition

Bayesian Updating

Analogy

A doctor listens to symptoms one at a time: Patient arrives → Prior: common cold is most likely. Symptom fever → Posterior shifts toward flu. Symptom rash → Posterior shifts further, now considering measles. Each symptom is a Bayes update: the posterior from the previous symptom becomes the prior for the next one.

Example

Doctors rarely compute explicit probabilities — they use pattern recognition and clinical experience. A Bayesian algorithm computes actual numbers. This contrast shows what Bayesian AI achieves that human intuition only approximates.

Example: Spam Filter (Naive Bayes)

A spam filter classifies emails using word frequencies. Prior: P(Spam) = 0.4, P(Ham) = 0.6. The email contains the word "winner":

P("winner" | Spam) = 0.8
P("winner" | Ham)  = 0.05

P(Spam | "winner") = (0.8 x 0.4) / (0.8 x 0.4 + 0.05 x 0.6)
                    = 0.32 / 0.35
                    ≈ 0.914 → 91.4% spam probability

If the email also contains "click," another Bayes update further increases the spam probability. Each word is a piece of evidence. This is Naive Bayes — "naive" because it assumes words are independent given the class. A simplification that works remarkably well in practice.

Python: Naive Bayes

from sklearn.naive_bayes import MultinomialNB

# Training features (word frequencies)
X_train = [[5, 1, 0], [4, 2, 0], [0, 1, 3], [1, 0, 4]]
y_train = ['spam', 'spam', 'ham', 'ham']

clf = MultinomialNB()
clf.fit(X_train, y_train)

# Classify a new email
clf.predict([[3, 1, 0]])  # → ['spam']

Caution

Misconception: The Prior Does Not Matter — Just Let the Data Speak

With large datasets, yes — the prior's influence fades. But with limited data (common in practice: medical imaging, rare events, small user bases), the prior can dominate the posterior. Choosing a thoughtful prior is not unfair bias — it is informed reasoning.

Training an AI model follows a similar pattern: the prior is the model's random initial weights. The evidence is the training data. The posterior is the trained model. Each training batch updates the weights — much like Bayes updates the posterior with each new piece of evidence. Strictly speaking, though, this is not Bayesian: ordinary training looks for a single good set of weights rather than a probability distribution, and a real prior lives in regularization (such as weight decay) rather than in the random weights. So Bayes helps you understand how AI learns — as a way of thinking about it, not as a blueprint.

Try it: The Coin Detective

The calculator above shows a single Bayes computation. Here you live the process: flip a hidden coin and watch your belief update flip by flip — from prior knowledge, through the evidence, to a new conviction (the posterior).

Note

Key Takeaways

  • P(A|B) ≠ P(B|A) — confusing the direction of conditioning is one of the most common probability errors; ignore the base rate as well and you fall for the Base Rate Fallacy.
  • Bayes' formula P(A|B) = P(B|A) x P(A) / P(B) combines prior knowledge (prior) with new observation (likelihood) to produce the updated belief (posterior).
  • Bayesian updating is iterative: each posterior becomes the new prior. With little data, the prior dominates; with lots of data, the evidence dominates. Spam filters and medical diagnostics use exactly this principle; AI training resembles it.

Check

Quiz: Bayes & Conditional Probability

What does P(A|B) mean?

Select one answer

Answer Key: 1) B · 2) B · 3) C · 4) D

Check

Checkpoint: Bayes & Conditional Probability

Can you explain this in your own words? Tick what you've got.

0 of 3 done