Naive Bayes (Classification)
Bayes plus a questionable independence assumption — and it works anyway.
Every word puts a weight on HAM or SPAM. The heavier side wins.
Spam-O-Meter
Analyze Email
Word Scanner
Known Words in Dictionary
What the probability journey shows
The spam meter is a horizontal scale: not spam on the left, spam on the right. A round marker travels along it while the filter reads the message word by word.
- What you see
- At the top a horizontal bar from HAM (green) to SPAM (red) with a round marker for the spam probability. Below it two stacks: on the left the words that point to harmless mail, on the right those that point to spam. The stronger a word, the taller its block.
- What happens
- The words of the message pass through the check one after another. Each word nudges the marker a little to the right if it looks suspicious, or to the left if it seems harmless. The marker jumps across the bar step by step until every word has had its turn.
- What you can do
- Pick a sample mail or type your own text, then start the journey with Start and halt it with Pause. Use Step to move word by word, and Reset to begin again. The base-rate slider shifts where the marker starts.
- What to watch for
- No single word decides on its own. The filter gathers many small clues and adds them up into one overall probability. Only where the marker comes to rest, left or right of the middle, is the verdict made.
Definition
What is Naive Bayes?
Analogy
Imagine you're an experienced mail sorter. After years of work, you know: Letters with 'FREE' and 'WINNER' usually end up in the trash.
Naive Bayes works the same way: The algorithm learns which words are typical for spam and which are typical for regular emails. For each new email, it checks the words and calculates: How likely is this spam?
The 'Naive' part means: We pretend the words have nothing to do with each other. This isn't quite correct, but works surprisingly well!
Definition
The Naive Bayes classifier is a probabilistic algorithm based on Bayes' Theorem (Thomas Bayes, 1763).
P(Spam|Words) = P(Words|Spam) × P(Spam) / P(Words)
The 'naive' conditional independence assumption enables likelihood factorization: P(W₁,W₂,...|C) = ∏ P(Wᵢ|C). Despite this simplifying assumption, the classifier achieves high accuracy in practice for text classification tasks.
This demo shows how a Naive Bayes spam filter analyzes emails word by word. Watch how the spam probability changes with each word.
Key Concepts
- Prior (Initial Belief): The base probability that any random email is spam - before we read it.
- Likelihood (Evidence Strength): How typical is each word for spam vs. legitimate emails?
- Posterior (Final Result): The updated spam probability after we've analyzed the words.
The 'Naive' Assumption"Naive" because the filter pretends all words are independent of each other. Rarely exactly true, yet it works surprisingly well in practice.
We pretend all words are independent of each other. 'FREE' and 'WINNER' are considered separately, even though they often appear together. This simplification makes calculation easy and still works well in practice.
Why Is It Called 'Spam'?
The term comes from a 1970 sketch by British comedy group Monty Python. In a café, a group of Vikings repeatedly chant the word 'SPAM' (a canned meat brand) until it drowns out all other conversations. Just like in the sketch, spam emails flood our inboxes with unwanted, repetitive messages.
The probabilities above are not magic — they were learned from real emails. Try it yourself: sort the examples below into Spam or Ham. Your filter builds itself from word counts, and you can test it right after.
CONGRATULATIONS! You have won 500,000 dollars in the lottery! Click here right now to claim your prize — urgent, free today only!
My crypto robot earns 10,000 dollars a day — guaranteed! Click immediately for free access. Top secret, do not share!
Exclusive offer — today only! Earn millions with our secret system. Start free immediately and secure your money!
Urgent: click here for a free credit. Guaranteed without checks, instantly available. Limited offer!
Hi team, the meeting tomorrow at 10 a.m. is confirmed. Please prepare the report for the project. Best regards
Hello, a quick question about the project: can we schedule a meeting for tomorrow? Please let me know briefly. Thanks
Dear team, the report for the customer is in the attachment. Discussion about it on Wednesday in the office. Greetings from accounting.
Hi colleague, the boss is asking about the document. Can you prepare it by tomorrow? Thanks for your work!
Bayes' Theorem
Naive Bayes is based on Bayes' Theorem by Thomas Bayes (1763): P(A|B) = P(B|A) × P(A) / P(B). It calculates the probability of a hypothesis (e.g., "Spam") given certain observations (e.g., words in an email).
The "naive" assumption: All features (words) are independent of each other. This isn't quite true - "free money" appears together more often than by chance. Yet the algorithm works surprisingly well!
Key Concepts
- Prior P(Spam): Base probability of spam (e.g., 20% of all emails)
- Likelihood P(Word|Spam): How likely does a word appear in spam?
- Evidence P(Word): How often does the word appear overall?
- Posterior P(Spam|Words): Final spam probability after analysis
Why 'Naive'?
The independence assumption simplifies calculation enormously: Instead of P(W₁,W₂,...|Spam), we simply calculate P(W₁|Spam) × P(W₂|Spam) × ... This enables fast classification even with many features.
Advantages
- Fast: Training and prediction are very efficient
- Low data requirements: Works well even with small datasets
- Interpretable: You can see which words contribute to the decision
- Robust: Insensitive to irrelevant features
Applications
Spam filters (Gmail, Outlook), sentiment analysis (positive/negative), document classification, medical diagnosis, recommendation systems, and speech recognition.
Try the demo! Enter an email and see step by step how the algorithm calculates the spam probability.
1
# Naive Bayes Spam Classifier
2
function classify_email(email):
3
# Prior: Base probability (from historical data)
4
prior_spam = 0.20 # 20% of all emails are spam
5
prior_ham = 0.80 # 80% are legitimate (ham)
6
7
# Extract words from email
8
words = email.text.lowercase().split()
9
known_words = filter(words, in_dictionary)
10
11
# Start with prior probabilities
12
prob_spam = prior_spam
13
prob_ham = prior_ham
14
15
for each word in known_words:
16
# 'Naive' assumption: Words are independent
17
likelihood_spam = dictionary[word].spam_probability
18
likelihood_ham = dictionary[word].ham_probability
19
20
# Multiply probabilities (log-sum in practice)
21
prob_spam = prob_spam × likelihood_spam
22
prob_ham = prob_ham × likelihood_ham
23
24
# Normalize: Probabilities must sum to 1
25
total = prob_spam + prob_ham
26
posterior_spam = prob_spam / total
27
28
# Make decision
29
if posterior_spam > 0.5:
30
return "SPAM"
31
else:
32
return "HAM"
📊 Set Prior
Set the base probability: What percentage of all emails are typically spam? This 'prior' probability comes from historical data (e.g., 20% spam, 80% ham).
prior_spam = 0.20 # 20% of all emails are spam
prior_ham = 0.80 # 80% are legitimate (ham)
📧 Email Input
A new email arrives and needs to be classified: Is it spam or a legitimate message (ham)?
🔤 Tokenization
Break down the text into individual words (tokens). Remove punctuation, convert to lowercase, filter known words.
📊 Apply Prior
Start with the base probability: What percentage of all emails are generally spam? This is our starting point.
📖 Calculate Likelihoods
For each word: Look up in the dictionary how typical it is for spam vs. ham. Multiply all likelihoods together.
🧮 Bayes' Theorem
Apply Bayes' theorem: Combine prior and likelihoods, normalize to 100%. Result: Posterior probability.
✅ Classification
Final decision: Posterior > 50% = spam, otherwise = ham. The email is sorted accordingly.
Check
Test Your Knowledge
What does 'Naive' mean in 'Naive Bayes'?
Select one answer
1. What does 'Naive' mean in 'Naive Bayes'?
2. What is the 'Prior' in Naive Bayes?
3. What happens when a very strong spam word like 'FREE' is found in an email?
4. Why does Naive Bayes work so well despite the 'naive' assumption?