Naive Bayes
Luis Serrano
Naive Bayes Classifier
Bayes Theorem
P(A B) =
P(B A)
P(B A)
P(A)
P(A)+ P(B Ac
)P(Ac
)
3. Bayes Theorem
Spam Detector
100 e-mails
Spam No spam
Spam Detector
25 75
Spam No spam
“Buy”
Spam Detector
25 75
Spam No spam
“Buy”
Spam Detector
25 75
Spam No spam
“Buy”
Spam Detector
25 75
Spam No spam
“Buy”
Spam Detector
Spam No spam
“Buy”
Spam Detector
Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
Spam No spam
“Buy”
Spam Detector
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
Spam No spam
“Buy”
Spam Detector
60%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
Spam No spam
“Buy”
Spam Detector
60%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
Spam No spam
“Buy”
Spam Detector
60%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20
60%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20 5
60%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20 580% 20%
60%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20 580% 20%
60%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20 580% 20%
60%
Solution:
80%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20 580% 20%
60%
Solution:
80%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
Spam No spam
“Buy”
Spam Detector
20 580% 20%
60%
Solution:
80%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
80%
Spam No spam
“Buy”
20 580% 20%
60%
Solution:
80%
80%
40%Quiz: If an e-mail
contains the word “buy”,
what is the probability
that it is spam?
100%
80%
Bayes Theorem
Spam No spam
Spam Detector
Spam No spam
Spam Detector
“Cheap”
Spam No spam
Spam Detector
“Cheap”
Spam No spam
Spam Detector
“Cheap”
Spam No spam
“Cheap”
Bayes Theorem
Spam No spam
“Cheap”
Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
Bayes Theorem
Spam No spam
“Cheap”
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
Bayes Theorem
Spam No spam
“Cheap”
60%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
15
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
15 10
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
15 1060% 40%
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
Solution:
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
15 1060% 40%
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
Solution:
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
15 1060% 40%
100%
Bayes Theorem
Spam No spam
“Cheap”
60%
Solution:
60%
80%
40%Quiz: If an e-mail
contains the word
“cheap”, what is the
probability that it is
spam?
15 1060% 40%
100%
60%
Bayes Theorem
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
Spam Detector
“Buy” and “Cheap”
Spam No spam
“Buy” and “Cheap”
Spam Detector
Spam No spam
“Buy” and “Cheap”
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
Spam Detector
Spam No spam
“Buy” and “Cheap”
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
Spam Detector
Spam No spam
“Buy” and “Cheap”
60%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
Spam Detector
Spam No spam
“Buy” and “Cheap”
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
Spam Detector
Spam No spam
“Buy” and “Cheap”
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
12
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
12
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
012
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
00%12100%
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
00%12100%
60%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
00%12100%
60%
Solution:
100%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
00%12100%
60%
Solution:
100%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
Spam Detector
Spam No spam
“Buy” and “Cheap”
00%12100%
60%
Solution:
100%
80%
40%Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
100%
100% ?
Spam Detector
Problem
Spam No spam
“Buy” and “Cheap”
Problem
Spam No spam
“Buy” and “Cheap”
12 e-mails
Problem
Spam No spam
“Buy” and “Cheap”
12 e-mails 0 e-mails?
Solution: Collect more data?
Solution: Collect more data?
Solution: Collect more data?
Spam Detector
Spam No spam
“Buy” and “Cheap”
12 e-mails 0 e-mails?
Spam Detector
Spam No spam
“Buy” and “Cheap”
12 e-mails 0 e-mails?
Guess?
Spam Detector
Spam Detector
100 e-mails
Spam Detector
100 e-mails
5 “Buy”
Spam Detector
100 e-mails
5 “Buy”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
10% “Cheap”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
10% “Cheap”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
10% “Cheap”
0.5% “Buy” and “Cheap”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
10% “Cheap”
0.5% “Buy” and “Cheap”
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
10% “Cheap”
0.5% “Buy” and “Cheap”
Independent
Spam Detector
100 e-mails
5 “Buy”
10 “Cheap”
5% “Buy”
10% “Cheap”
0.5% “Buy” and “Cheap”
Independent
That’s
naive!
Spam
Spam Detector
Spam
Spam Detector
25 e-mails
Spam
Spam Detector
25 e-mails
Spam
Spam Detector
25 e-mails
20 “Buy”
Spam
Spam Detector
25 e-mails
20 “Buy” 4/5
Spam
Spam Detector
25 e-mails
20 “Buy” 4/5
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
3/5
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
3/5
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
3/5
12/25
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
3/5
12/25 x 25
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
3/5
12/25 = 12 “Buy” and “Cheap”x 25
Spam
Spam Detector
25 e-mails
20 “Buy”
15 Cheap
4/5
3/5
12/25 = 12 “Buy” and “Cheap”x 25
No spam
Spam Detector
No spam
Spam Detector
75 e-mails
No spam
Spam Detector
75 e-mails
5 “Buy”
No spam
Spam Detector
75 e-mails
5 “Buy” 1/15
No spam
Spam Detector
75 e-mails
5 “Buy”
10 “Cheap”
1/15
No spam
Spam Detector
75 e-mails
5 “Buy”
10 “Cheap”
1/15
2/15
No spam
Spam Detector
75 e-mails
5 “Buy”
10 “Cheap”
1/15
2/15
2/225
No spam
Spam Detector
75 e-mails
5 “Buy”
10 “Cheap”
1/15
2/15
2/225 x 75
No spam
Spam Detector
75 e-mails
5 “Buy”
10 “Cheap”
1/15
2/15
2/225 = 2/3 “Buy” and “Cheap”x 75
No spam
Spam Detector
75 e-mails
5 “Buy”
10 “Cheap”
1/15
2/15
2/225 = 2/3 “Buy” and “Cheap”x 75
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12 2/312
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/312
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
=
38
36
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
=
38
36
= 94.737%
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
=
38
36
= 94.737%
94.737%
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
5.263% =
38
36
= 94.737%
94.737%
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
Spam No spam
Spam Detector
“Buy” and “Cheap”
12 2/3
5.263% =
38
36
= 94.737%
94.737%
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
94.737%
Spam No spam
“Buy” and “Cheap”
12 2/3
5.263% =
38
36
= 94.737%
94.737%
Quiz: If an e-mail
contains the words “buy”
and “cheap”, what is the
probability that it is
spam?
12
2/31212 + 2/3
12
Naive Bayes Classifier
94.737%
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20 4/5
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20 4/5 5
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20 4/5 5 1/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5 5 1/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
5 1/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
5
10
1/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
12 + 2/3
12
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
=
38
36
12 + 2/3
12
Naive Bayes
Spam No spam
Total
Buy
Cheap
Buy & Cheap
25 75
20
15
12
4/5
3/5
12/25
5
10
2/3
1/15
2/15
2/225
=
38
36
= 94.737%
12 + 2/3
12
Naive Bayes
Spam No spam
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 30
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 4/15 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 4/15 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 4/15 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 12/125 4/15 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
12/5 + 4/15
12/5
12/5 12/125 4/15 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
=
40
36
12/5 + 4/15
12/5
12/5 12/125 4/15 12/3375
Naive Bayes
Total
Buy 20 4/5 5 1/15
Cheap 15 3/5 10 2/15
Work
Buy, Cheap,
& Work
Spam No Spam
25 75
5 1/5 30 6/15
=
40
36
= 90%
12/5 + 4/15
12/5
12/5 12/125 4/15 12/3375
Naive Bayes
4. The math
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
20
25
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
20
25
25
100
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
20
25
25
100 +
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
20
25
25
100
5
75+
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
20
25
25
100
5
75
75
100+
Bayes Theorem
P(S B) =
P(B S)
P(B S)
P(S)
P(S)+P(B H)P(H)
S: Spam
H: Ham (not spam)
B: ‘Buy’
P(spam if “Buy”) =
20
25
25
100
20
25
25
100
5
75
75
100+
= 80%
Naive Bayes
P(“Buy” & “Cheap”) = P(“Buy”) P(“Cheap”)
Naive Bayes
P(“Buy” & “Cheap”) = P(“Buy”) P(“Cheap”)
P(B C) = P(B) P(C)
U
Naive Bayes
P(“Buy” & “Cheap”) = P(“Buy”) P(“Cheap”)
Naive
P(B C) = P(B) P(C)
U
Naive Bayes
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
15
25
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
25
100
15
25
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
25
100
+
15
25
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
25
100
5
75
+
15
25
15
25
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
25
100
5
75
+
15
25
15
25
10
75
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
25
100
5
75
75
100
+
15
25
15
25
10
75
= 94.737%
Naive BayesS: Spam
H: Ham (not spam)
B: ‘Buy’
C: ‘Cheap’
P(S B C) =
P(S)
P(S) + P(H)
U P(B C S)
U
P(B C S)
U
P(B C H)
U
P(B S)P(C S)
P(B S)P(C S) P(B H)P(C H)
P(spam if “Buy” & “Cheap”) =
20
25
25
100
20
25
25
100
5
75
75
100
+
15
25
15
25
10
75
Thank you!
Thank you!
Subscribe, like,
share, comment
youtube.com/c/LuisSerrano
Thank you!
@luis_likes_math
Subscribe, like,
share, comment
youtube.com/c/LuisSerrano

Bayes theorem and Naive Bayes algorithm