Naive Bayes Introduction

Lesson 12 of 19 in Coddy's Introduction to Machine Learning course.

Naive Bayes is a simple yet powerful algorithm used for classification tasks in machine learning.

The Naive Bayes algorithm is based on Bayes' theorem, which tells us how to calculate the probability of an event (like an email being spam) based on prior knowledge of conditions that might be related to the event. The "naive" part comes from the assumption that all the features (like the words in the email) are independent of each other, which simplifies the calculations. Being independent means one thing happening doesn't change the chances of another thing happening.

Source: Wikipedia, A Naive Bayes Classifier updating its estimate while more data is fed to it.
Here's how it works:

Count Occurrences: First, count how many times each data point appears in different classes. In email spam detection we'll count how many times each word appeared in spam or not spam emails, in medical diagnosis, we might count how many times (or how long) each symptom appears.
Calculate Probabilities: For a new data point calculate the probability of belonging to each class(what is the probability of a new email being spam and what is the probability of being not spam)
Decision: Finally, the algorithm combines all these probabilities using Bayes' theorem and decides whether the email is more likely to be spam or not spam. The higher the probability, the more likely it belong to this class.

The cool thing about Naive Bayes is its fast and easy implementation. It works well with high-dimensional data (like text) and can handle missing values. However, the independence assumption isn't always true in real life, which can sometimes lead to less accurate results compared to more complex algorithms.

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

Challenge

Medium

In this challenge, we'll learn how to calculate the probability of being in a particular class. The dataset play_outside.csv contains the following columns: temperature, mood, motivation, windy, play. Each row represents the same person and whether he decided to play outside or not based on features (temperature, mood, motivation, windy). Our goal is to predict whether he will go play outside based on features (temperature, mood, motivation, windy).

Here is an example of a dataset:

temperature	mood	motivation	windy	play
low	good	high	no	yes
moderate	good	high	no	yes
low	low	medium	yes	no

The following equations might look intimidating but we'll break them down.
Naive Bayes formula: P(Class|Feature) = P(Feature|Class) * P(Class) / P(Feature)

P(Feature|Class) -Likelihood
P(Class) - Prior probability of the class
P(Feature) - Prior probability of the predictor
P(Class|Feature) - In other words, What is the probability of the unseen row being classified to class x.

To solve this we'll need to calculate each of the components of the formula (P(B|Class) * P(Class) / P(B))

P(Class) - Prior probability of class
This equals the number of occurrences of each class, divided by the total number of classes. For example in our 3-rows example, there are 2/3 yes and 1/3 no.
P(yes) = 2/3
P(no) = 1/3
P(Feature|Class) -Likelihood
This equals the number of occurrences of each feature of a given class. For example
P(temperature = low|play = yes) - There are two rows where play = yes, and one row where temperature = low (of only the rows where play = yes), so this equals 1/2.
P(Feature)
This equals the number of occurrences of each feature divided by the total number of rows. For example:
p(motivation=high) = 2/3 because there are two rows with high motivation divided by the total number of rows (3)

The challenge

You are given two dictionaries probability_features, probability_target and the dataset_name

Calculate for each feature (column) the probability of each attribute.
For example here is the value of probability_target :

probability_target = {
    "yes": 0.66667, # (2/3)
    "no": 0.33333 # (1/3)   
}

Here is the value of probability_features:

probability_features = {
	"temperature": {
		"low": {
			"yes": 0.5, # (1/2)
			"no": 0.5, # (1/2)
		},
		"moderate": {
			"yes": 1, # (1/1)
			"no": 0, # (0/1)
		}
	}
}

Keep in mind that the feature names and the attribute names are not constant. Each dataset has different features and different names.

I would recommend using the pandas module to make the calculations easier.

Try it yourself

import pandas as pd
probability_features = {}
probability_target = {}
dataset_path = input()
target_variable = input()
df = pd.read_csv(dataset_path)
# Write your code here


# -------------------- Sort and print --------------------
# Don't change below this line
sort_features = {q: {k: v for k, v in sorted(a.items() if a != None else {}, key=lambda item: item[0])} for q, a in sorted(probability_features.items(), key=lambda item: item[0])}
sort_target = {k: v for k, v in sorted(probability_target.items(), key=lambda item: item[0])}
print(sort_features)
print(sort_target)