Naive Bayes (Classification)
Bayes plus a questionable independence assumption — and it works anyway.
What is Naive Bayes?
Imagine you're an experienced mail sorter. After years of work, you know: Letters with 'FREE' and 'WINNER' usually end up in the trash.
Naive Bayes works the same way: The algorithm learns which words are typical for spam and which are typical for regular emails. For each new email, it checks the words and calculates: How likely is this spam?
The 'Naive' part means: We pretend the words have nothing to do with each other. This isn't quite correct, but works surprisingly well!
Analogy:
Imagine you're an experienced mail sorter. After years of work, you know: Letters with 'FREE' and 'WINNER' usually end up in the trash.
Naive Bayes works the same way: The algorithm learns which words are typical for spam and which are typical for regular emails. For each new email, it checks the words and calculates: How likely is this spam?
The 'Naive' part means: We pretend the words have nothing to do with each other. This isn't quite correct, but works surprisingly well!
Definition:
The Naive Bayes classifier is a probabilistic algorithm based on Bayes' Theorem (Thomas Bayes, 1763).
P(Spam|Words) = P(Words|Spam) × P(Spam) / P(Words)
The 'naive' conditional independence assumption enables likelihood factorization: P(W₁,W₂,...|C) = ∏ P(Wᵢ|C). Despite this simplifying assumption, the classifier achieves high accuracy in practice for text classification tasks.
How This Demo Works
This demo shows how a Naive Bayes spam filter analyzes emails word by word. Watch how the spam probability changes with each word.
Key Concepts
- Prior (Initial Belief):The base probability that any random email is spam - before we read it.
- Likelihood (Evidence Strength):How typical is each word for spam vs. legitimate emails?
- Posterior (Final Result):The updated spam probability after we've analyzed the words.
The 'Naive' Assumption
We pretend all words are independent of each other. 'FREE' and 'WINNER' are considered separately, even though they often appear together. This simplification makes calculation easy and still works well in practice.
Why Is It Called 'Spam'?
The term comes from a 1970 sketch by British comedy group Monty Python. In a café, a group of Vikings repeatedly chant the word 'SPAM' (a canned meat brand) until it drowns out all other conversations. Just like in the sketch, spam emails flood our inboxes with unwanted, repetitive messages.
Train your own spam filter
The probabilities above are not magic — they were learned from real emails. Try it yourself: sort the examples below into Spam or Ham. Your filter builds itself from word counts, and you can test it right after.
CONGRATULATIONS! You have won 500,000 dollars in the lottery! Click here right now to claim your prize — urgent, free today only!
My crypto robot earns 10,000 dollars a day — guaranteed! Click immediately for free access. Top secret, do not share!
Exclusive offer — today only! Earn millions with our secret system. Start free immediately and secure your money!
Urgent: click here for a free credit. Guaranteed without checks, instantly available. Limited offer!
Hi team, the meeting tomorrow at 10 a.m. is confirmed. Please prepare the report for the project. Best regards
Hello, a quick question about the project: can we schedule a meeting for tomorrow? Please let me know briefly. Thanks
Dear team, the report for the customer is in the attachment. Discussion about it on Wednesday in the office. Greetings from accounting.
Hi colleague, the boss is asking about the document. Can you prepare it by tomorrow? Thanks for your work!
Spam Detector
What the probability journey shows
The spam meter is a horizontal scale: not spam on the left, spam on the right. A round marker travels along it while the filter reads the message word by word.
- What you see
- A horizontal bar, green on the left for harmless mail, red on the right for spam, with a scale from 0 to 100 percent in between. A round marker sits on it and shifts its colour from green through yellow to red, depending on where it stands.
- What happens
- The words of the message pass through the check one after another. Each word nudges the marker a little to the right if it looks suspicious, or to the left if it seems harmless. The marker jumps across the bar step by step until every word has had its turn.
- What you can do
- Pick a sample mail or type your own text, then start the journey with Start and halt it with Pause. Use Step to move word by word, and Reset to begin again. The base-rate slider shifts where the marker starts.
- What to watch for
- No single word decides on its own. The filter gathers many small clues and adds them up into one overall probability. Only where the marker comes to rest, left or right of the middle, is the verdict made.
Spam-O-Meter
Analyze Email
Controls
Word Scanner
Known Words in Dictionary
Naive Bayes Explained
Bayes' Theorem
Naive Bayes is based on Bayes' Theorem by Thomas Bayes (1763): P(A|B) = P(B|A) × P(A) / P(B). It calculates the probability of a hypothesis (e.g., "Spam") given certain observations (e.g., words in an email).
The "naive" assumption: All features (words) are independent of each other. This isn't quite true - "free money" appears together more often than by chance. Yet the algorithm works surprisingly well!
Key Concepts
- Prior P(Spam): Base probability of spam (e.g., 20% of all emails)
- Likelihood P(Word|Spam): How likely does a word appear in spam?
- Evidence P(Word): How often does the word appear overall?
- Posterior P(Spam|Words): Final spam probability after analysis
Why 'Naive'?
The independence assumption simplifies calculation enormously: Instead of P(W₁,W₂,...|Spam), we simply calculate P(W₁|Spam) × P(W₂|Spam) × ... This enables fast classification even with many features.
Advantages
- Fast: Training and prediction are very efficient
- Low data requirements: Works well even with small datasets
- Interpretable: You can see which words contribute to the decision
- Robust: Insensitive to irrelevant features
Applications
Spam filters (Gmail, Outlook), sentiment analysis (positive/negative), document classification, medical diagnosis, recommendation systems, and speech recognition.
Try the demo! Enter an email and see step by step how the algorithm calculates the spam probability.
1
# Naive Bayes Spam Classifier
2
function classify_email(email):
3
# Prior: Base probability (from historical data)
4
prior_spam = 0.20 # 20% of all emails are spam
5
prior_ham = 0.80 # 80% are legitimate (ham)
6
7
# Extract words from email
8
words = email.text.lowercase().split()
9
known_words = filter(words, in_dictionary)
10
11
# Start with prior probabilities
12
prob_spam = prior_spam
13
prob_ham = prior_ham
14
15
for each word in known_words:
16
# 'Naive' assumption: Words are independent
17
likelihood_spam = dictionary[word].spam_probability
18
likelihood_ham = dictionary[word].ham_probability
19
20
# Multiply probabilities (log-sum in practice)
21
prob_spam = prob_spam × likelihood_spam
22
prob_ham = prob_ham × likelihood_ham
23
24
# Normalize: Probabilities must sum to 1
25
total = prob_spam + prob_ham
26
posterior_spam = prob_spam / total
27
28
# Make decision
29
if posterior_spam > 0.5:
30
return "SPAM"
31
else:
32
return "HAM"
📊 Set Prior
Set the base probability: What percentage of all emails are typically spam? This 'prior' probability comes from historical data (e.g., 20% spam, 80% ham).
prior_spam = 0.20 # 20% of all emails are spam
prior_ham = 0.80 # 80% are legitimate (ham)
📧 Email Input
A new email arrives and needs to be classified: Is it spam or a legitimate message (ham)?
🔤 Tokenization
Break down the text into individual words (tokens). Remove punctuation, convert to lowercase, filter known words.
📊 Apply Prior
Start with the base probability: What percentage of all emails are generally spam? This is our starting point.
📖 Calculate Likelihoods
For each word: Look up in the dictionary how typical it is for spam vs. ham. Multiply all likelihoods together.
🧮 Bayes' Theorem
Apply Bayes' theorem: Combine prior and likelihoods, normalize to 100%. Result: Posterior probability.
✅ Classification
Final decision: Posterior > 50% = spam, otherwise = ham. The email is sorted accordingly.
Test Your Knowledge
What does 'Naive' mean in 'Naive Bayes'?
1. What does 'Naive' mean in 'Naive Bayes'?
- ☐ A) The algorithm is simple to understand
- ☐ B) The assumption that all words are independent of each other
- ☐ C) The algorithm often makes mistakes
- ☐ D) It's a simplified version of more complex algorithms
2. What is the 'Prior' in Naive Bayes?
- ☐ A) The probability of a specific word
- ☐ B) The result of the classification
- ☐ C) The base probability of spam before we've analyzed the email
- ☐ D) The number of spam words in the email
3. What happens when a very strong spam word like 'FREE' is found in an email?
- ☐ A) The spam probability increases significantly
- ☐ B) The email is immediately marked as spam
- ☐ C) All other words are ignored
- ☐ D) The prior is set to 100%
4. Why does Naive Bayes work so well despite the 'naive' assumption?
- ☐ A) Because words actually are independent of each other
- ☐ B) Because only few words are analyzed
- ☐ C) Because the prior is always correctly set
- ☐ D) Because for classification, the relative ranking of probabilities matters more than exact values
Related Content
Article
Bayes & Conditional Probability
Conditional probability: the tool that identifies statisticians — they calculate differently.
Bias & Data Quality
Bad data in, bad AI out — with the uncomfortable punchline that there is no "perfectly fair".
Measures of Central Tendency: Where Is the Middle?
Three ways to find the "center" of data — and the entertaining question of which one is being dishonest right now.
Correlation vs. Causation
Why every statistician flinches when someone says "correlates with" and means "causes".
Distributions: The Shape of Data
The shape of data explained — and why a bell curve is rarer than you think.
Rules & Logic: Expert Systems
AI before it learned from data: asked experts, wrote down rules, hoped.
Safety & Fraud Protection
What to do when the voice on the phone sounds like a relative but isn't one.
Linear & Logistic Regression
The mathematical foundation that every deep learning course only gets to after three hours.
Programming vs. Training
How programming changed when people stopped writing every rule themselves.
How Good Is Your Model? Metrics That Actually Matter
Evaluating models without self-deception — metrics that do more than just look good.
When the Model Memorizes (Overfitting)
How to notice that the model didn't learn but memorized.
Probability & Expected Value
Expected value: the average of futures, weighted by probability.
Supervised Learning — Learning with a Teacher
Supervised Learning: the ML paradigm where someone diligently labeled things beforehand.
Demo
Decision Tree
Interactive decision-tree demo: set points, tune depth, watch splits appear live and experience overfitting.
Perceptron (Neural Networks)
Discover the first artificial neuron - the Big Bang of machine learning from 1957.
Supervised Learning
Join Sharlock Helmes in his cleverest case: learning to distinguish between genuine clues and Moriarty's sophisticated red herrings. Elementary, my dear algorithm!