Case Study (Social Network Ads) :

SKLearrn (Naive Bayes(GaussianNB))

- Naïve Bayes:

    - A classification technique based on Bayes’ Theorem

- Example: 

    - Classify a new client as eligible to retire or not based on age and salary

    - The probability of customer being eligible to retire given her/his features


            - P(Retire|X) = P(X|Retire) * P(Retire) / P(X)


- Bayes’ Rule

    - combines prior probability and likelihood to form a posterior probability

    - Posterior probability = (Prior probability * Likelihood)

            - Prior probabilities -> P(Retire)

            - Likelihood -> P(x|Retire) 

- Calculate Posterior probability = (Prior probability * Likelihood)

        - Posterior probability (Red)  = 20/60 * 3/20 = 1/20

        - Posterior probability (Blue) = 40/60 * 1/40 = 1/60


        - Posterior probability (Red) 1/20 > Posterior probability (Blue) 1/60

        - Result : X classified as Red (not eligible to retire) 


- Marginal Probability P(X)

    - Divide Posterior probability by Marginal Probability

              - To Adjust the equation

              - To come up with a number ranges between 0 & 1


- Marginal Probability = No of observations in circle/total observations = 4/60

       - Posterior probability(Red)/Marginal Probability  =  (1/20)/(4/60) = 0.75

       - Posterior probability(Blue)/Marginal Probability =  (1/60)/(4/60) = 0.25

Overview

- Importing the Relevant Libraries

- Loading the Data

- Declaring the Dependent and the Independent variables

- Splitting the dataset into the Training set and Test set

- Feature Scaling

- Training the Naive Bayes Model (GaussianNB)

- Predicting the Test Set Results

- Confusion Matrix

- Classification Report

- K-Fold Cross Validation

- Visualising the Training Set Results

- Visualising the Test Set Results

Importing the Relevant Libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()

Loading the Data

url = "https://DataScienceSchools.github.io/Machine_Learning/Classification_Models_Intuition/Social_Network_Ads.csv"

df = pd.read_csv(url)

df.head()

	Age	EstimatedSalary	Purchased
0	19	19000	0
1	35	20000	0
2	26	43000	0
3	27	57000	0
4	19	76000	0

Declaring the Dependent & the Independent Variables

X = df.iloc[:, :-1].values

y = df.iloc[:, -1].values

Splitting the Dataset into the Training Set and Test Set

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20, random_state = 4)

Feature Scaling

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()

X_train = sc.fit_transform(X_train)

X_test = sc.transform(X_test)

Training the Naive Bayes Model (GaussianNB)

from sklearn.naive_bayes import GaussianNB

model = GaussianNB()

model.fit(X_train, y_train)

GaussianNB(priors=None, var_smoothing=1e-09)

Predicting the Test Set Results

y_pred = model.predict(X_test)

Confusion Matrix

   - A confusion matrix used to describe the performance of a classification model

   - TP (True Positive): Model predicted Correctly
   - TN (True Negative): Model predicted Correctly
   - FP (Flase Positive): Model predicted True but it is actually False

       - Type I Error
       - Predicting people have cancer, but actually they do not have cancer
       - Predictiong earthquake will happen, but it actually does not happen

   - FN (False Negative): Model predicted False but it is actually True 

       - Type II Error -> Life-threatening Error (Must avoid it at all cost)
       - Predicting people do not have cancer, but actually they have
       - Predictiong earthquake will not happen, but it actually happens

- Accuracy = Correct/Total 
- Error Rate = Wrong/Total 

from sklearn.metrics import confusion_matrix, accuracy_score

cm = confusion_matrix(y_test, y_pred)

accuracy = accuracy_score(y_test, y_pred)

print("Accuracy is: {:.2f} %".format(accuracy*100))

sns.heatmap(cm, annot=True, fmt='d')

plt.show()

Accuracy is: 92.50 %

Classification Report

from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred))

              precision    recall  f1-score   support

           0       0.98      0.92      0.95        59
           1       0.80      0.95      0.87        21

    accuracy                           0.93        80
   macro avg       0.89      0.93      0.91        80
weighted avg       0.93      0.93      0.93        80

k-Fold Cross Validation

   - Accuracy of test set is often a misleading metric

   - A solution to this problem is a procedure called cross-validation

   - k-fold cross-validation is used to evaluate machine learning models

   - How the performance measure is calculated by k-fold cross-validation? 

        1. The training set is split into k smaller sets  

        1. Each set is used as training data to train the model

        2. The remaining part of the data used as a test set to compute the accuracy 

        3. Then the average of all accuracies is calculated & reported

from sklearn.model_selection import cross_val_score

accuracies = cross_val_score(estimator = model, X = X_train, y = y_train, cv = 10)

print("Accuracy: {:.2f} %".format(accuracies.mean()*100))

print("Standard Deviation: {:.2f} %".format(accuracies.std()*100))

Accuracy: 88.44 %
Standard Deviation: 4.65 %

Visualising the Training Set Results

from matplotlib.colors import ListedColormap

X_set, y_set = sc.inverse_transform(X_train), y_train

X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 10, stop = X_set[:, 0].max() + 10, step = 0.25),
                     np.arange(start = X_set[:, 1].min() - 1000, stop = X_set[:, 1].max() + 1000, step = 0.25))

plt.contourf(X1, X2, model.predict(sc.transform(np.array([X1.ravel(), X2.ravel()]).T)).reshape(X1.shape),
             alpha = 0.75, cmap = ListedColormap(('magenta', 'blue')))

plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())

for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1], color = ListedColormap(('magenta', 'blue'))(i), label = j)

plt.title('Naive Bayes (Training set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()

Visualising the Test Set Results

from matplotlib.colors import ListedColormap

X_set, y_set = sc.inverse_transform(X_test), y_test

X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 10, stop = X_set[:, 0].max() + 10, step = 0.25),
                     np.arange(start = X_set[:, 1].min() - 1000, stop = X_set[:, 1].max() + 1000, step = 0.25))

plt.contourf(X1, X2, model.predict(sc.transform(np.array([X1.ravel(), X2.ravel()]).T)).reshape(X1.shape),
             alpha = 0.75, cmap = ListedColormap(('magenta', 'blue')))

plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())

for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1], color = ListedColormap(('magenta', 'blue'))(i), label = j)

plt.title('Naive Bayes (Test set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()