Logistic Regression and Error Metrics¶

📚 Contents¶

📘 Logistic Regression (theory)
🧪 Classification and Hypothesis Testing Practice Project: Travel Package Purchase Prediction
⚙️ Model Training
📊 Exploratory Data Analysis
🧹 Data Preparation for Modeling
🎯 Model Evaluation Criterion
🧠 Building the Model: Logistic Regression
📈 Evaluation Metrics
🔍 Model Building and Assessment

Logistic Regression (theory)¶

What it models. Probability of the positive class (y=1) via the sigmoid: $$ p(y=1\mid \mathbf{x})=\sigma(z)=\frac{1}{1+e^{-z}},\quad z=\beta_0+\beta_1 x_1+\cdots+\beta_p x_p. $$

Log-odds (logit). $$ \mathrm{logit}(p)=\log\!\left(\frac{p}{1-p}\right)=\beta_0+\sum_j \beta_j x_j. $$

Training. Estimate $\beta$ by maximizing likelihood (equivalently minimizing logistic loss) using numerical optimization (liblinear, lbfgs, etc.).

Interpreting coefficients. A one-unit increase in $(x_j)$ multiplies the odds by $(e^{\beta_j})$. Standardizing features helps compare magnitudes.

Decision threshold. The model outputs probabilities (\hat p). We convert them to labels using a threshold (t) (often 0.5). Moving (t) trades off recall vs precision.

In [4]:

            
                Copied!
                
                    
                    
                
                

        
import numpy as np
import matplotlib.pyplot as plt

# Create the data
z = np.linspace(-10, 10, 200)
sigmoid = 1 / (1 + np.exp(-z))

# Plot
plt.figure(figsize=(7,5))
plt.plot(z, sigmoid, label=r'$\sigma(z) = \frac{1}{1+e^{-z}}$', linewidth=2)
plt.axvline(0, color='gray', linestyle='--')
plt.axhline(0.5, color='red', linestyle='--', label='Decision threshold (0.5)')
plt.title("🧩 Logistic (Sigmoid) Function", fontsize=14)
plt.xlabel("z (Linear combination of features)")
plt.ylabel("Probability p(y=1 | x)")
plt.legend()
plt.grid(True)
plt.show()
import numpy as np
import matplotlib.pyplot as plt

# Create the data
z = np.linspace(-10, 10, 200)
sigmoid = 1 / (1 + np.exp(-z))

# Plot
plt.figure(figsize=(7,5))
plt.plot(z, sigmoid, label=r'$\sigma(z) = \frac{1}{1+e^{-z}}$', linewidth=2)
plt.axvline(0, color='gray', linestyle='--')
plt.axhline(0.5, color='red', linestyle='--', label='Decision threshold (0.5)')
plt.title("🧩 Logistic (Sigmoid) Function", fontsize=14)
plt.xlabel("z (Linear combination of features)")
plt.ylabel("Probability p(y=1 | x)")
plt.legend()
plt.grid(True)
plt.show()

/Users/sebastianazocarmondaca/Library/Python/3.11/lib/python/site-packages/IPython/core/pylabtools.py:170: UserWarning: Glyph 129513 (\N{JIGSAW PUZZLE PIECE}) missing from font(s) DejaVu Sans.
  fig.canvas.print_figure(bytes_io, **kw)

In [5]:

            
                Copied!
                
                    
                    
                
                

        
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression

# Create a synthetic dataset
X, y = make_classification(n_features=2, n_redundant=0, n_informative=2,
                           random_state=2, n_clusters_per_class=1)
model = LogisticRegression()
model.fit(X, y)

# Build a grid for the decision surface
xx, yy = np.meshgrid(np.linspace(X[:,0].min()-1, X[:,0].max()+1, 100),
                     np.linspace(X[:,1].min()-1, X[:,1].max()+1, 100))
Z = model.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1].reshape(xx.shape)

# Plot
plt.figure(figsize=(7,5))
plt.contourf(xx, yy, Z, cmap='RdBu', alpha=0.6)
plt.scatter(X[:,0], X[:,1], c=y, cmap='bwr', edgecolor='k')
plt.title("🎯 Logistic Regression Decision Boundary", fontsize=14)
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.colorbar(label="Probability of y=1")
plt.show()
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression

# Create a synthetic dataset
X, y = make_classification(n_features=2, n_redundant=0, n_informative=2,
                           random_state=2, n_clusters_per_class=1)
model = LogisticRegression()
model.fit(X, y)

# Build a grid for the decision surface
xx, yy = np.meshgrid(np.linspace(X[:,0].min()-1, X[:,0].max()+1, 100),
                     np.linspace(X[:,1].min()-1, X[:,1].max()+1, 100))
Z = model.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1].reshape(xx.shape)

# Plot
plt.figure(figsize=(7,5))
plt.contourf(xx, yy, Z, cmap='RdBu', alpha=0.6)
plt.scatter(X[:,0], X[:,1], c=y, cmap='bwr', edgecolor='k')
plt.title("🎯 Logistic Regression Decision Boundary", fontsize=14)
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.colorbar(label="Probability of y=1")
plt.show()

/Users/sebastianazocarmondaca/Library/Python/3.11/lib/python/site-packages/IPython/core/pylabtools.py:170: UserWarning: Glyph 127919 (\N{DIRECT HIT}) missing from font(s) DejaVu Sans.
  fig.canvas.print_figure(bytes_io, **kw)

Classification and Hypothesis Testing Practice Project: Travel Package Purchase Prediction¶

Context¶

You are a Data Scientist for a tourism company named "Visit with us". The Policy Maker of the company wants to enable and establish a viable business model to expand the customer base. A viable business model is a central concept that helps you to understand the existing ways of doing the business and how to change the ways for the benefit of the tourism sector.

One of the ways to expand the customer base is to introduce a new offering of packages. Currently, there are 5 types of packages the company is offering - Basic, Standard, Deluxe, Super Deluxe, King. Looking at the data of the last year, we observed that 18% of the customers purchased the packages. However, it was difficult to identify the potential customers because customers were contacted at random without looking at the available information.

The company is now planning to launch a new product i.e. Wellness Tourism Package. Wellness Tourism is defined as Travel that allows the traveler to maintain, enhance or kick-start a healthy lifestyle, and support or increase one's sense of well-being. This time company wants to harness the available data of existing and potential customers to target the right customers.

You as a Data Scientist at "Visit with us" travel company has to analyze the customers' data and information to provide recommendations to the Policy Maker and build a model to predict the potential customer who is going to purchase the newly introduced travel package. The model will be built to make predictions before a customer is contacted.

Objective¶

To build a model to predict which customer is potentially going to purchase the newly introduced travel package.

Data Description¶

CustomerID: Unique customer ID
ProdTaken: Whether the customer has purchased a package or not (0: No, 1: Yes)
Age: Age of customer
TypeofContact: How customer was contacted (Company Invited or Self Inquiry)
CityTier: City tier depends on the development of a city, population, facilities, and living standards. The categories are ordered i.e. Tier 1 > Tier 2 > Tier 3. It's the city the customer lives in.
DurationOfPitch: Duration of the pitch by a salesperson to the customer
Occupation: Occupation of customer
Gender: Gender of customer
NumberOfPersonVisiting: Total number of persons planning to take the trip with the customer
NumberOfFollowups: Total number of follow-ups has been done by the salesperson after the sales pitch
ProductPitched: Product pitched by the salesperson
PreferredPropertyStar: Preferred hotel property rating by customer
MaritalStatus: Marital status of customer
NumberOfTrips: Average number of trips in a year by customer
Passport: The customer has a passport or not (0: No, 1: Yes)
PitchSatisfactionScore: Sales pitch satisfaction score
OwnCar: Whether the customers own a car or not (0: No, 1: Yes)
NumberOfChildrenVisiting: Total number of children with age less than 5 planning to take the trip with the customer
Designation: Designation of the customer in the current organization
MonthlyIncome: Gross monthly income of the customer

Importing the libraries required¶

Model Training¶

We will train a logistic regression in four steps:

Data prep: train/test split and any scaling/feature engineering needed.
Fit the model: estimate (\beta) by maximizing likelihood.
Predict probabilities: obtain (\hat p = P(y=1\mid \mathbf{x})).
Classify: apply a threshold (e.g., 0.5) to turn (\hat p) into labels.

In [ ]:

            
                Copied!
                
                    
                    
                
                

        
# Importing the basic libraries we will require for the project

# Libraries to help with reading and manipulating data
import pandas as pd
import numpy as np

# Libaries to help with data visualization
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()

# Importing the Machine Learning models we require from Scikit-Learn
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
from sklearn.ensemble import RandomForestClassifier

# Importing the other functions we may require from Scikit-Learn
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import MinMaxScaler, LabelEncoder, OneHotEncoder
from sklearn.impute import SimpleImputer

# To get diferent metric scores
from sklearn.metrics import confusion_matrix,classification_report,roc_auc_score,precision_recall_curve,roc_curve,make_scorer

# Code to ignore warnings from function usage
import warnings;
import numpy as np
warnings.filterwarnings('ignore')
# Importing the basic libraries we will require for the project

# Libraries to help with reading and manipulating data
import pandas as pd
import numpy as np

# Libaries to help with data visualization
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()

# Importing the Machine Learning models we require from Scikit-Learn
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
from sklearn.ensemble import RandomForestClassifier

# Importing the other functions we may require from Scikit-Learn
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import MinMaxScaler, LabelEncoder, OneHotEncoder
from sklearn.impute import SimpleImputer

# To get diferent metric scores
from sklearn.metrics import confusion_matrix,classification_report,roc_auc_score,precision_recall_curve,roc_curve,make_scorer

# Code to ignore warnings from function usage
import warnings;
import numpy as np
warnings.filterwarnings('ignore')

Loading the dataset¶

In [ ]:

            
                Copied!
                
import pandas as pd 
# Loading the dataset - sheet_name parameter is used if there are Basicple tabs in the excel file.
data=pd.read_excel("https://raw.githubusercontent.com/hizocar/usm-course-intensivo/main/docs/datasets/Tourism.xlsx",sheet_name='Tourism')
import pandas as pd 
# Loading the dataset - sheet_name parameter is used if there are Basicple tabs in the excel file.
data=pd.read_excel("https://raw.githubusercontent.com/hizocar/usm-course-intensivo/main/docs/datasets/Tourism.xlsx",sheet_name='Tourism')

Overview of the dataset¶

View the first and last 5 rows of the dataset¶

Let's view the first few rows and last few rows of the dataset in order to understand its structure a little better.

We will use the head() and tail() methods from Pandas to do this.

In [ ]:

            
                Copied!
                
data.head()
data.head()

Out[ ]:

	CustomerID	ProdTaken	Age	TypeofContact	CityTier	DurationOfPitch	Occupation	Gender	NumberOfPersonVisiting	NumberOfFollowups	ProductPitched	PreferredPropertyStar	MaritalStatus	NumberOfTrips	Passport	PitchSatisfactionScore	OwnCar	NumberOfChildrenVisiting	Designation	MonthlyIncome
0	200000	1	41.0	Self Enquiry	3	6.0	Salaried	Female	3	3.0	Deluxe	3.0	Single	1.0	1	2	1	0.0	Manager	20993.0
1	200001	0	49.0	Company Invited	1	14.0	Salaried	Male	3	4.0	Deluxe	4.0	Divorced	2.0	0	3	1	2.0	Manager	20130.0
2	200002	1	37.0	Self Enquiry	1	8.0	Free Lancer	Male	3	4.0	Basic	3.0	Single	7.0	1	3	0	0.0	Executive	17090.0
3	200003	0	33.0	Company Invited	1	9.0	Salaried	Female	2	3.0	Basic	3.0	Divorced	2.0	1	5	1	1.0	Executive	17909.0
4	200004	0	NaN	Self Enquiry	1	8.0	Small Business	Male	2	3.0	Basic	4.0	Divorced	1.0	0	5	1	0.0	Executive	18468.0

In [ ]:

            
                Copied!
                
data.tail()
data.tail()

Out[ ]:

	CustomerID	ProdTaken	Age	TypeofContact	CityTier	DurationOfPitch	Occupation	Gender	NumberOfPersonVisiting	NumberOfFollowups	ProductPitched	PreferredPropertyStar	MaritalStatus	NumberOfTrips	Passport	PitchSatisfactionScore	OwnCar	NumberOfChildrenVisiting	Designation	MonthlyIncome
4883	204883	1	49.0	Self Enquiry	3	9.0	Small Business	Male	3	5.0	Deluxe	4.0	Unmarried	2.0	1	1	1	1.0	Manager	26576.0
4884	204884	1	28.0	Company Invited	1	31.0	Salaried	Male	4	5.0	Basic	3.0	Single	3.0	1	3	1	2.0	Executive	21212.0
4885	204885	1	52.0	Self Enquiry	3	17.0	Salaried	Female	4	4.0	Standard	4.0	Married	7.0	0	1	1	3.0	Senior Manager	31820.0
4886	204886	1	19.0	Self Enquiry	3	16.0	Small Business	Male	3	4.0	Basic	3.0	Single	3.0	0	5	0	2.0	Executive	20289.0
4887	204887	1	36.0	Self Enquiry	1	14.0	Salaried	Male	4	4.0	Basic	4.0	Unmarried	3.0	1	3	1	2.0	Executive	24041.0

Understand the shape of the dataset¶

In [ ]:

            
                Copied!
                
data.shape
data.shape

Out[ ]:

(4888, 20)

The dataset has 4888 rows and 20 columns.

Check the data types of the columns for the dataset¶

In [ ]:

            
                Copied!
                
data.info()
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4888 entries, 0 to 4887
Data columns (total 20 columns):
 #   Column                    Non-Null Count  Dtype  
---  ------                    --------------  -----  
 0   CustomerID                4888 non-null   int64  
 1   ProdTaken                 4888 non-null   int64  
 2   Age                       4662 non-null   float64
 3   TypeofContact             4863 non-null   object 
 4   CityTier                  4888 non-null   int64  
 5   DurationOfPitch           4637 non-null   float64
 6   Occupation                4888 non-null   object 
 7   Gender                    4888 non-null   object 
 8   NumberOfPersonVisiting    4888 non-null   int64  
 9   NumberOfFollowups         4843 non-null   float64
 10  ProductPitched            4888 non-null   object 
 11  PreferredPropertyStar     4862 non-null   float64
 12  MaritalStatus             4888 non-null   object 
 13  NumberOfTrips             4748 non-null   float64
 14  Passport                  4888 non-null   int64  
 15  PitchSatisfactionScore    4888 non-null   int64  
 16  OwnCar                    4888 non-null   int64  
 17  NumberOfChildrenVisiting  4822 non-null   float64
 18  Designation               4888 non-null   object 
 19  MonthlyIncome             4655 non-null   float64
dtypes: float64(7), int64(7), object(6)
memory usage: 763.9+ KB

We can see that 8 columns have less than 4,888 non-null values i.e. columns have missing values.

Check the percentage of missing values in each column¶

In [ ]:

            
                Copied!
                
pd.DataFrame(data={'% of Missing Values':round(data.isna().sum()/data.isna().count()*100,2)}).sort_values(by='% of Missing Values',ascending=False)
pd.DataFrame(data={'% of Missing Values':round(data.isna().sum()/data.isna().count()*100,2)}).sort_values(by='% of Missing Values',ascending=False)

Out[ ]:

	% of Missing Values
DurationOfPitch	5.14
MonthlyIncome	4.77
Age	4.62
NumberOfTrips	2.86
NumberOfChildrenVisiting	1.35
NumberOfFollowups	0.92
PreferredPropertyStar	0.53
TypeofContact	0.51
Designation	0.00
OwnCar	0.00
PitchSatisfactionScore	0.00
Passport	0.00
CustomerID	0.00
MaritalStatus	0.00
ProdTaken	0.00
NumberOfPersonVisiting	0.00
Gender	0.00
Occupation	0.00
CityTier	0.00
ProductPitched	0.00

DurationOfPitch column has 5.14% missing values out of the total observations.
The MonthlyIncome column has 4.77% missing values out of the total observations.
The Age column has 4.62% missing values out of the total observations.
TypeofContact column has 0.51% missing values out of the total observations.
The NumberOfFollowups column has 0.92% missing values out of the total observations.
PreferredPropertyStar column has 0.53% missing values out of the total observations.
NumberOfTrips column has 2.86% missing values out of the total observations.
NumberOfChildrenVisiting column has 1.35% missing values out of the total observations.
We will impute these values after we split the data into train and test sets.

Check the number of unique values in each column¶

In [ ]:

            
                Copied!
                
data.nunique()
data.nunique()

Out[ ]:

CustomerID                  4888
ProdTaken                      2
Age                           44
TypeofContact                  2
CityTier                       3
DurationOfPitch               34
Occupation                     4
Gender                         3
NumberOfPersonVisiting         5
NumberOfFollowups              6
ProductPitched                 5
PreferredPropertyStar          3
MaritalStatus                  4
NumberOfTrips                 12
Passport                       2
PitchSatisfactionScore         5
OwnCar                         2
NumberOfChildrenVisiting       4
Designation                    5
MonthlyIncome               2475
dtype: int64

We can drop the column - CustomerID as it is unique for each customer and will not add value to the model.
Most of the variables are categorical except - Age, duration of pitch, monthly income, and number of trips of customers.

Dropping the unique values column

In [ ]:

            
                Copied!
                
# Dropping CustomerID column
data.drop(columns='CustomerID',inplace=True)
# Dropping CustomerID column
data.drop(columns='CustomerID',inplace=True)

Question 1: Check the summary statistics of the dataset and write your observations (2 Marks)¶

Let's check the statistical summary of the data.

In [ ]:

            
                Copied!
                
data.describe().T
data.describe().T

Out[ ]:

	count	mean	std	min	25%	50%	75%	max
ProdTaken	4888.0	0.188216	0.390925	0.0	0.0	0.0	0.0	1.0
Age	4662.0	37.622265	9.316387	18.0	31.0	36.0	44.0	61.0
CityTier	4888.0	1.654255	0.916583	1.0	1.0	1.0	3.0	3.0
DurationOfPitch	4637.0	15.490835	8.519643	5.0	9.0	13.0	20.0	127.0
NumberOfPersonVisiting	4888.0	2.905074	0.724891	1.0	2.0	3.0	3.0	5.0
NumberOfFollowups	4843.0	3.708445	1.002509	1.0	3.0	4.0	4.0	6.0
PreferredPropertyStar	4862.0	3.581037	0.798009	3.0	3.0	3.0	4.0	5.0
NumberOfTrips	4748.0	3.236521	1.849019	1.0	2.0	3.0	4.0	22.0
Passport	4888.0	0.290917	0.454232	0.0	0.0	0.0	1.0	1.0
PitchSatisfactionScore	4888.0	3.078151	1.365792	1.0	2.0	3.0	4.0	5.0
OwnCar	4888.0	0.620295	0.485363	0.0	0.0	1.0	1.0	1.0
NumberOfChildrenVisiting	4822.0	1.187267	0.857861	0.0	1.0	1.0	2.0	3.0
MonthlyIncome	4655.0	23619.853491	5380.698361	1000.0	20346.0	22347.0	25571.0	98678.0

Write your Answer here :

Mean and median of age column are very close to each other i.e. approx 37 and 36 respectively.
Duration of pitch has some outliers at the right end as the 75th percentile value is 20 and the max value is 127. We need to explore this further.
It seems like monthly income has some outliers at both ends. We need to explore this further.
The number of trips also has some outliers as the 75th percentile value is 4 and the max value is 22.
We can see that the target variable - ProdTaken is imbalanced as most of the values are 0.

Check the count of each unique category in each of the categorical variables.¶

In [ ]:

            
                Copied!
                
                    
                    
                
                

        
# Making a list of all catrgorical variables
cat_col=['TypeofContact', 'CityTier','Occupation', 'Gender', 'NumberOfPersonVisiting',
       'NumberOfFollowups', 'ProductPitched', 'PreferredPropertyStar',
       'MaritalStatus', 'Passport', 'PitchSatisfactionScore',
       'OwnCar', 'NumberOfChildrenVisiting', 'Designation']

# Printing number of count of each unique value in each column
for column in cat_col:
    print(data[column].value_counts())
    print('-'*50)
# Making a list of all catrgorical variables
cat_col=['TypeofContact', 'CityTier','Occupation', 'Gender', 'NumberOfPersonVisiting',
       'NumberOfFollowups', 'ProductPitched', 'PreferredPropertyStar',
       'MaritalStatus', 'Passport', 'PitchSatisfactionScore',
       'OwnCar', 'NumberOfChildrenVisiting', 'Designation']

# Printing number of count of each unique value in each column
for column in cat_col:
    print(data[column].value_counts())
    print('-'*50)

Self Enquiry       3444
Company Invited    1419
Name: TypeofContact, dtype: int64
--------------------------------------------------
1    3190
3    1500
2     198
Name: CityTier, dtype: int64
--------------------------------------------------
Salaried          2368
Small Business    2084
Large Business     434
Free Lancer          2
Name: Occupation, dtype: int64
--------------------------------------------------
Male       2916
Female     1817
Fe Male     155
Name: Gender, dtype: int64
--------------------------------------------------
3    2402
2    1418
4    1026
1      39
5       3
Name: NumberOfPersonVisiting, dtype: int64
--------------------------------------------------
4.0    2068
3.0    1466
5.0     768
2.0     229
1.0     176
6.0     136
Name: NumberOfFollowups, dtype: int64
--------------------------------------------------
Basic           1842
Deluxe          1732
Standard         742
Super Deluxe     342
King             230
Name: ProductPitched, dtype: int64
--------------------------------------------------
3.0    2993
5.0     956
4.0     913
Name: PreferredPropertyStar, dtype: int64
--------------------------------------------------
Married      2340
Divorced      950
Single        916
Unmarried     682
Name: MaritalStatus, dtype: int64
--------------------------------------------------
0    3466
1    1422
Name: Passport, dtype: int64
--------------------------------------------------
3    1478
5     970
1     942
4     912
2     586
Name: PitchSatisfactionScore, dtype: int64
--------------------------------------------------
1    3032
0    1856
Name: OwnCar, dtype: int64
--------------------------------------------------
1.0    2080
2.0    1335
0.0    1082
3.0     325
Name: NumberOfChildrenVisiting, dtype: int64
--------------------------------------------------
Executive         1842
Manager           1732
Senior Manager     742
AVP                342
VP                 230
Name: Designation, dtype: int64
--------------------------------------------------

The Free lancer category in the occupation column has just 2 entries out of 4,888 observations.
We can see that Gender has 3 unique values which include - 'Fe Male' and 'Female'. This must be a data input error, we should replace 'Fe Male' with 'Female'.
NumberOfPersonVisiting equal to 5 has a count equal to 3 only.
The majority of the customers are married.
The majority of the customers own a car.

In [ ]:

            
                Copied!
                
# Replacing 'Fe Male' with 'Female'
data.Gender=data.Gender.replace('Fe Male', 'Female')
# Replacing 'Fe Male' with 'Female'
data.Gender=data.Gender.replace('Fe Male', 'Female')

In [ ]:

            
                Copied!
                
# Converting the data type of each categorical variable to 'category'
for column in cat_col:
    data[column]=data[column].astype('category')
# Converting the data type of each categorical variable to 'category'
for column in cat_col:
    data[column]=data[column].astype('category')

In [ ]:

            
                Copied!
                
data.info()
data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4888 entries, 0 to 4887
Data columns (total 19 columns):
 #   Column                    Non-Null Count  Dtype   
---  ------                    --------------  -----   
 0   ProdTaken                 4888 non-null   int64   
 1   Age                       4662 non-null   float64 
 2   TypeofContact             4863 non-null   category
 3   CityTier                  4888 non-null   category
 4   DurationOfPitch           4637 non-null   float64 
 5   Occupation                4888 non-null   category
 6   Gender                    4888 non-null   category
 7   NumberOfPersonVisiting    4888 non-null   category
 8   NumberOfFollowups         4843 non-null   category
 9   ProductPitched            4888 non-null   category
 10  PreferredPropertyStar     4862 non-null   category
 11  MaritalStatus             4888 non-null   category
 12  NumberOfTrips             4748 non-null   float64 
 13  Passport                  4888 non-null   category
 14  PitchSatisfactionScore    4888 non-null   category
 15  OwnCar                    4888 non-null   category
 16  NumberOfChildrenVisiting  4822 non-null   category
 17  Designation               4888 non-null   category
 18  MonthlyIncome             4655 non-null   float64 
dtypes: category(14), float64(4), int64(1)
memory usage: 260.3 KB

In [ ]:

            
                Copied!
                
df = data.copy()
df = data.copy()

Exploratory Data Analysis¶

Question 2: Univariate Analysis¶

Let's explore these variables in some more depth by observing their distributions.

We will first define a hist_box() function that provides both a boxplot and a histogram in the same visual, with which we can perform univariate analysis on the columns of this dataset.

In [ ]:

            
                Copied!
                
                    
                    
                
                

        
# Defining the hist_box() function
def hist_box(data, col):
    f, (ax_box, ax_hist) = plt.subplots(2, sharex=True, gridspec_kw={'height_ratios': (0.15, 0.85)}, figsize=(12, 6))
    # Adding a graph in each part
    sns.boxplot(data=data, x=col, ax=ax_box, showmeans=True)
    sns.histplot(data=data, x=col, kde=True, ax=ax_hist)
    plt.show()
# Defining the hist_box() function
def hist_box(data, col):
    f, (ax_box, ax_hist) = plt.subplots(2, sharex=True, gridspec_kw={'height_ratios': (0.15, 0.85)}, figsize=(12, 6))
    # Adding a graph in each part
    sns.boxplot(data=data, x=col, ax=ax_box, showmeans=True)
    sns.histplot(data=data, x=col, kde=True, ax=ax_hist)
    plt.show()

# Question 2.1: Plot the histogram and box plot for the variable `Age` using the hist_box function provided and write your insights. (1 Mark)¶

In [ ]:

            
                Copied!
                
hist_box(df, "Age")
hist_box(df, "Age")

Write your Answer here :

Age distribution looks approximately normally distributed.
The boxplot for the age column confirms that there are no outliers for this variable
Age can be an important variable while targeting customers for the tourism package. We will further explore this in bivariate analysis.

# Question 2.2: Plot the histogram and box plot for the variable `Duration of Pitch` using the hist_box function provided and write your insights. (1 Mark)¶

In [ ]:

            
                Copied!
                
hist_box(df, 'DurationOfPitch')
hist_box(df, 'DurationOfPitch')

Write your Answer here :

The distribution for the duration of pitch is right-skewed.
The duration of the pitch for most of the customers is less than 20 minutes.
There are some observations that can be considered as outliers as they are very far from the upper whisker in the boxplot. Let's check how many such extreme values are there.

In [ ]:

            
                Copied!
                
df[df['DurationOfPitch']>40]
df[df['DurationOfPitch']>40]

Out[ ]:

	ProdTaken	Age	TypeofContact	CityTier	DurationOfPitch	Occupation	Gender	NumberOfPersonVisiting	NumberOfFollowups	ProductPitched	PreferredPropertyStar	MaritalStatus	NumberOfTrips	Passport	PitchSatisfactionScore	OwnCar	NumberOfChildrenVisiting	Designation	MonthlyIncome
1434	0	NaN	Company Invited	3	126.0	Salaried	Male	2	3.0	Basic	3.0	Married	3.0	0	1	1	1.0	Executive	18482.0
3878	0	53.0	Company Invited	3	127.0	Salaried	Male	3	4.0	Basic	3.0	Married	4.0	0	1	1	2.0	Executive	22160.0

We can see that there are just two observations which can be considered as outliers.

Note:

This code uses boolean indexing in Pandas. Assuming 'df' is a DataFrame, df['DurationOfPitch'] > 40 creates a boolean mask where each row is checked against the condition 'DurationOfPitch' greater than 40, resulting in a series of True/False values.

When this boolean series is used to index the DataFrame df[], it returns only the rows where the condition is True, effectively filtering out rows where the 'DurationOfPitch' is not greater than 40.

Lets plot the histogram and box plot for the variable Monthly Income using the hist_box function

In [ ]:

            
                Copied!
                
hist_box(df, 'MonthlyIncome')
hist_box(df, 'MonthlyIncome')

The distribution for monthly income shows that most of the values lie between 20,000 to 40,000.
Income is one of the important factors to consider while approaching a customer with a certain package. We can explore this further in bivariate analysis.
There are some observations on the left and some observations on the right of the boxplot which can be considered as outliers. Let's check how many such extreme values are there.

In [ ]:

            
                Copied!
                
df[(df.MonthlyIncome>40000) | (df.MonthlyIncome<12000)]
df[(df.MonthlyIncome>40000) | (df.MonthlyIncome<12000)]

Out[ ]:

	Age	TypeofContact	CityTier	DurationOfPitch	Occupation	Gender	NumberOfPersonVisiting	NumberOfFollowups	ProductPitched	PreferredPropertyStar	MaritalStatus	NumberOfTrips	Passport	PitchSatisfactionScore	OwnCar	NumberOfChildrenVisiting	Designation	MonthlyIncome
38	36.0	Self Enquiry	1	11.0	Salaried	Female	2	4.0	Basic	NaN	Divorced	1.0	1	2	1	0.0	Executive	95000.0
142	38.0	Self Enquiry	1	9.0	Large Business	Female	2	3.0	Deluxe	3.0	Single	4.0	1	5	0	0.0	Manager	1000.0
2482	37.0	Self Enquiry	1	12.0	Salaried	Female	3	5.0	Basic	5.0	Divorced	2.0	1	2	1	1.0	Executive	98678.0
2586	39.0	Self Enquiry	1	10.0	Large Business	Female	3	4.0	Deluxe	3.0	Single	5.0	1	5	0	1.0	Manager	4678.0

There are just four such observations which can be considered as outliers.

Lets plot the histogram and box plot for the variable Number of Trips using the hist_box function

In [ ]:

            
                Copied!
                
hist_box(df,'NumberOfTrips')
hist_box(df,'NumberOfTrips')

The distribution for the number of trips is right-skewed
Boxplot shows that the number of trips has some outliers at the right end. Let's check how many such extreme values are there.

In [ ]:

            
                Copied!
                
df.NumberOfTrips.value_counts(normalize=True)
df.NumberOfTrips.value_counts(normalize=True)

Out[ ]:

2.0     0.308340
3.0     0.227254
1.0     0.130581
4.0     0.100674
5.0     0.096462
6.0     0.067818
7.0     0.045914
8.0     0.022115
19.0    0.000211
21.0    0.000211
20.0    0.000211
22.0    0.000211
Name: NumberOfTrips, dtype: float64

We can see that most of the customers i.e. 52% have taken 2 or 3 trips.
As expected, with the increase in the number of trips the percentage of customers is decreasing.
The percentage of categories 19 or above is very less. We can consider these values as outliers.
We can see that there are just four observations with a number of trips 19 or greater

In [ ]:

            
                Copied!
                
df.NumberOfTrips.value_counts(normalize=True).sort_index()
df.NumberOfTrips.value_counts(normalize=True).sort_index()

Out[ ]:

1.0     0.130581
2.0     0.308340
3.0     0.227254
4.0     0.100674
5.0     0.096462
6.0     0.067818
7.0     0.045914
8.0     0.022115
19.0    0.000211
20.0    0.000211
21.0    0.000211
22.0    0.000211
Name: NumberOfTrips, dtype: float64

Removing these outliers form duration of pitch, monthly income, and number of trips.

In [ ]:

            
                Copied!
                
# Dropping observaions with duration of pitch greater than 40. There are just 2 such observations
df.drop(index=df[df.DurationOfPitch>40].index,inplace=True)

# Dropping observation with monthly income less than 12000 or greater than 40000. There are just 4 such observations
df.drop(index=df[(df.MonthlyIncome>40000) | (df.MonthlyIncome<12000)].index,inplace=True)

# Dropping observations with number of trips greater than 8. There are just 4 such observations
df.drop(index=df[df.NumberOfTrips>8].index,inplace=True)
# Dropping observaions with duration of pitch greater than 40. There are just 2 such observations
df.drop(index=df[df.DurationOfPitch>40].index,inplace=True)

# Dropping observation with monthly income less than 12000 or greater than 40000. There are just 4 such observations
df.drop(index=df[(df.MonthlyIncome>40000) | (df.MonthlyIncome<12000)].index,inplace=True)

# Dropping observations with number of trips greater than 8. There are just 4 such observations
df.drop(index=df[df.NumberOfTrips>8].index,inplace=True)

# Let's understand the distribution of the categorical variables¶

Number of Person Visiting

In [ ]:

            
                Copied!
                
sns.countplot(x = df['NumberOfPersonVisiting'])
plt.show()
sns.countplot(x = df['NumberOfPersonVisiting'])
plt.show()

In [ ]:

            
                Copied!
                
df['NumberOfPersonVisiting'].value_counts(normalize=True)
df['NumberOfPersonVisiting'].value_counts(normalize=True)

Out[ ]:

3    0.491390
2    0.289873
4    0.210127
1    0.007995
5    0.000615
Name: NumberOfPersonVisiting, dtype: float64

Most customers have 3 persons who are visiting with them. This can be because most people like to travel with family.
As mentioned earlier, there are just 3 observations where the number of persons visiting with the customers are 5 i.e. 0.1%.

Occupation

In [ ]:

            
                Copied!
                
sns.countplot(x = df['Occupation'])
plt.show()
sns.countplot(x = df['Occupation'])
plt.show()

In [ ]:

            
                Copied!
                
df['Occupation'].value_counts(normalize=True)
df['Occupation'].value_counts(normalize=True)

Out[ ]:

Salaried          0.484215
Small Business    0.427224
Large Business    0.088151
Free Lancer       0.000410
Name: Occupation, dtype: float64

The majority of customers i.e. 91% are either salaried or owns a small business.
As mentioned earlier, the freelancer category has only 2 observations.

City Tier

In [ ]:

            
                Copied!
                
sns.countplot(x = df['CityTier'])
plt.show()
sns.countplot(x = df['CityTier'])
plt.show()

In [ ]:

            
                Copied!
                
df['CityTier'].value_counts(normalize=True)
df['CityTier'].value_counts(normalize=True)

Out[ ]:

1    0.652317
3    0.307093
2    0.040590
Name: CityTier, dtype: float64

Most of the customers i.e. approx 65% are from tier 1 cities. This can be because of better living standards and exposure as compared to tier 2 and tier 3 cities.
Surprisingly, tier 3 cities have a much higher count than tier 2 cities. This can be because the company has less marketing in tier 2 cities.

Gender

In [ ]:

            
                Copied!
                
sns.countplot(x = df['Gender'])
plt.show()
sns.countplot(x = df['Gender'])
plt.show()

In [ ]:

            
                Copied!
                
df['Gender'].value_counts(normalize=True)
df['Gender'].value_counts(normalize=True)

Out[ ]:

Male      0.596556
Female    0.403444
Name: Gender, dtype: float64

Male customers are more than the number of female customers
There are approx 60% male customers as compared to 40% female customers
This might be because males do the booking/inquiry when traveling with females which imply that males are the direct customers of the company.

Number of Follow ups

In [ ]:

            
                Copied!
                
sns.countplot(x = df['NumberOfFollowups'])
plt.show()
sns.countplot(x = df['NumberOfFollowups'])
plt.show()

In [ ]:

            
                Copied!
                
df['NumberOfFollowups'].value_counts(normalize=True).sort_index()
df['NumberOfFollowups'].value_counts(normalize=True).sort_index()

Out[ ]:

1.0    0.036416
2.0    0.047383
3.0    0.302504
4.0    0.426857
5.0    0.158701
6.0    0.028140
Name: NumberOfFollowups, dtype: float64

We can see that company usually follow-ups with 3 or 4 times with their customers
We can explore this further and observe which number of follow-ups have more customers who buy the product.

Product Pitched

In [ ]:

            
                Copied!
                
sns.countplot(x = df['ProductPitched'])
plt.show()
sns.countplot(x = df['ProductPitched'])
plt.show()

In [ ]:

            
                Copied!
                
df['ProductPitched'].value_counts(normalize=True)
df['ProductPitched'].value_counts(normalize=True)

Out[ ]:

Basic           0.376384
Deluxe          0.354244
Standard        0.152112
Super Deluxe    0.070111
King            0.047150
Name: ProductPitched, dtype: float64

The company pitches Deluxe or Basic packages to their customers more than the other packages.
This might be because the company makes more profit from Deluxe or Basic packages or these packages are less expensive, so preferred by the majority of the customers.

Type of Contact

In [ ]:

            
                Copied!
                
sns.countplot(x = df['TypeofContact'])
plt.show()
sns.countplot(x = df['TypeofContact'])
plt.show()

In [ ]:

            
                Copied!
                
df['TypeofContact'].value_counts(normalize=True)
df['TypeofContact'].value_counts(normalize=True)

Out[ ]:

Self Enquiry       0.70884
Company Invited    0.29116
Name: TypeofContact, dtype: float64

There are approx 70% of customers who reached out to the company first i.e. self-inquiry.
This shows the positive outreach of the company as most of the inquires are initiated from the customer's end.

Designation

In [ ]:

            
                Copied!
                
sns.countplot(x = df['Designation'])
plt.show()
sns.countplot(x = df['Designation'])
plt.show()

In [ ]:

            
                Copied!
                
df['Designation'].value_counts(normalize=True)
df['Designation'].value_counts(normalize=True)

Out[ ]:

Executive         0.376384
Manager           0.354244
Senior Manager    0.152112
AVP               0.070111
VP                0.047150
Name: Designation, dtype: float64

Approx 73% of the customers are at the executive or manager level.
We can see that the higher the position, the lesser number of observations which makes sense as executives/managers are more common than AVP/VP.

Product Taken

In [ ]:

            
                Copied!
                
sns.countplot(x = df['ProdTaken'])
plt.show()
sns.countplot(x = df['ProdTaken'])
plt.show()

In [ ]:

            
                Copied!
                
df['ProdTaken'].value_counts(normalize=True)
df['ProdTaken'].value_counts(normalize=True)

Out[ ]:

0    0.811808
1    0.188192
Name: ProdTaken, dtype: float64

This plot shows the distribution of both classes in the target variable is imbalanced.
We only have approx 19% of customers who have purchased the product.

Question 3: Bivariate Analysis¶

# Question 3.1: Find and visualize the correlation matrix using a heatmap and write your observations from the plot. (2 Marks)¶

In [ ]:

            
                Copied!
                
cols_list = df.select_dtypes(include=np.number).columns.tolist()

plt.figure(figsize=(12, 7))
sns.heatmap(data[cols_list].corr(), annot=True, vmin=-1, vmax=1, fmt=".2f", cmap="Spectral")
plt.show()
cols_list = df.select_dtypes(include=np.number).columns.tolist()

plt.figure(figsize=(12, 7))
sns.heatmap(data[cols_list].corr(), annot=True, vmin=-1, vmax=1, fmt=".2f", cmap="Spectral")
plt.show()

Write your Answer here :

The Number of trips and age have a weak positive correlation, which makes sense as age increases number of trips is expected to increase slightly.
Age and monthly income are positively correlated.
ProdTaken has a weak negative correlation with age which agrees with our earlier observation that as age increases the probability for purchasing a package decreases.
No other variables have a high correlation among them.

We will define a stacked barplot() function to help analyse how the target variable varies across predictor categories.

In [ ]:

            
                Copied!
                
# Defining the stacked_barplot() function
def stacked_barplot(data,predictor,target,figsize=(10,6)):
  (pd.crosstab(data[predictor],data[target],normalize='index')*100).plot(kind='bar',figsize=figsize,stacked=True)
  plt.legend(loc="lower right")
  plt.ylabel(target)
# Defining the stacked_barplot() function
def stacked_barplot(data,predictor,target,figsize=(10,6)):
  (pd.crosstab(data[predictor],data[target],normalize='index')*100).plot(kind='bar',figsize=figsize,stacked=True)
  plt.legend(loc="lower right")
  plt.ylabel(target)

# Question 3.2: Plot the stacked barplot for the variable `Marital Status` against the target variable `ProdTaken` using the stacked_barplot function provided and write your insights. (1 Mark)¶

In [ ]:

            
                Copied!
                
stacked_barplot(data, "MaritalStatus", "ProdTaken" )
stacked_barplot(data, "MaritalStatus", "ProdTaken" )

Write your Answer here :

The married people are the most common customer for the company but this graph shows that the conversion rate is higher for single and unmarried customers as compared to the married customers.
The company can target single and unmarried customers more and can modify packages as per these customers.

# Question 3.3: Plot the stacked barplot for the variable `ProductPitched` against the target variable `ProdTaken` using the stacked_barplot function provided and write your insights. (1 Mark)¶

In [ ]:

            
                Copied!
                
stacked_barplot(df, "ProductPitched", "ProdTaken" )
stacked_barplot(df, "ProductPitched", "ProdTaken" )

Write your Answer here :

The conversion rate of customers is higher if the product pitched is Basic. This might be because the basic package is less expensive.
We saw earlier that company pitches the deluxe package more than the standard package, but the standard package shows a higher conversion rate than the deluxe package. The company can pitch standard packages more often.

Let's plot the stacked barplot for the variable Passport against the target variable ProdTaken using the stacked_barplot function.

In [ ]:

            
                Copied!
                
stacked_barplot(data, "Passport", "ProdTaken" )
stacked_barplot(data, "Passport", "ProdTaken" )

The conversion rate for customers with a passport is higher as compared to the customers without a passport.
The company should customize more international packages to attract more such customers.

Let's plot the stacked barplot for the variable Designation against the target variable ProdTaken using the stacked_barplot function.

In [ ]:

            
                Copied!
                
stacked_barplot(data, "Designation", "ProdTaken" )
stacked_barplot(data, "Designation", "ProdTaken" )

The conversion rate of executives is higher than other designations.
Customers at VP and AVP positions have the least conversion rate.

Data Preparation for Modeling¶

Separating the independent variables (X) and the dependent variable (Y)

In [ ]:

            
                Copied!
                
# Separating target variable and other variables
X=data.drop(columns='ProdTaken')
Y=data['ProdTaken']
# Separating target variable and other variables
X=data.drop(columns='ProdTaken')
Y=data['ProdTaken']

As we aim to predict customers who are more likely to buy the product, we should drop the columns DurationOfPitch', 'NumberOfFollowups', 'ProductPitched', 'PitchSatisfactionScore' as these columns would not be available at the time of prediction for new data.

In [ ]:

            
                Copied!
                
# Dropping columns
X.drop(columns=['DurationOfPitch','NumberOfFollowups','ProductPitched','PitchSatisfactionScore'],inplace=True)
# Dropping columns
X.drop(columns=['DurationOfPitch','NumberOfFollowups','ProductPitched','PitchSatisfactionScore'],inplace=True)

Splitting the data into a 70% train and 30% test set

Some classification problems can exhibit a large imbalance in the distribution of the target classes: for instance there could be several times more negative samples than positive samples. In such cases it is recommended to use the stratified sampling technique to ensure that relative class frequencies are approximately preserved in each train and validation fold.

In [ ]:

            
                Copied!
                
# Splitting the data into train and test sets
X_train,X_test,y_train,y_test=train_test_split(X,Y,test_size=0.30,random_state=1,stratify=Y)
# Splitting the data into train and test sets
X_train,X_test,y_train,y_test=train_test_split(X,Y,test_size=0.30,random_state=1,stratify=Y)

As we saw earlier, our data has missing values. We will impute missing values using median for continuous variables and mode for categorical variables. We will use SimpleImputer to do this.

The SimpleImputer provides basic strategies for imputing missing values. Missing values can be imputed with a provided constant value, or using the statistics (mean, median, or most frequent) of each column in which the missing values are located.

In [ ]:

            
                Copied!
                
si1=SimpleImputer(strategy='median')

median_imputed_col=['Age','MonthlyIncome','NumberOfTrips']

# Fit and transform the train data
X_train[median_imputed_col]=si1.fit_transform(X_train[median_imputed_col])

#Transform the test data i.e. replace missing values with the median calculated using training data
X_test[median_imputed_col]=si1.transform(X_test[median_imputed_col])
si1=SimpleImputer(strategy='median')

median_imputed_col=['Age','MonthlyIncome','NumberOfTrips']

# Fit and transform the train data
X_train[median_imputed_col]=si1.fit_transform(X_train[median_imputed_col])

#Transform the test data i.e. replace missing values with the median calculated using training data
X_test[median_imputed_col]=si1.transform(X_test[median_imputed_col])

Then, transform is used on the test data (X_test) to replace missing values in the same columns with the medians calculated from the training data. This step is crucial because you're using the statistics calculated from the training data to transform the test data, ensuring consistency between the two datasets.

In [ ]:

            
                Copied!
                
si2=SimpleImputer(strategy='most_frequent')

mode_imputed_col=['TypeofContact','PreferredPropertyStar','NumberOfChildrenVisiting']

# Fit and transform the train data
X_train[mode_imputed_col]=si2.fit_transform(X_train[mode_imputed_col])

# Transform the test data i.e. replace missing values with the mode calculated using training data
X_test[mode_imputed_col]=si2.transform(X_test[mode_imputed_col])
si2=SimpleImputer(strategy='most_frequent')

mode_imputed_col=['TypeofContact','PreferredPropertyStar','NumberOfChildrenVisiting']

# Fit and transform the train data
X_train[mode_imputed_col]=si2.fit_transform(X_train[mode_imputed_col])

# Transform the test data i.e. replace missing values with the mode calculated using training data
X_test[mode_imputed_col]=si2.transform(X_test[mode_imputed_col])

In [ ]:

            
                Copied!
                
# Checking that no column has missing values in train or test sets
print(X_train.isna().sum())
print('-'*30)
print(X_test.isna().sum())
# Checking that no column has missing values in train or test sets
print(X_train.isna().sum())
print('-'*30)
print(X_test.isna().sum())

Age                         0
TypeofContact               0
CityTier                    0
Occupation                  0
Gender                      0
NumberOfPersonVisiting      0
PreferredPropertyStar       0
MaritalStatus               0
NumberOfTrips               0
Passport                    0
OwnCar                      0
NumberOfChildrenVisiting    0
Designation                 0
MonthlyIncome               0
dtype: int64
------------------------------
Age                         0
TypeofContact               0
CityTier                    0
Occupation                  0
Gender                      0
NumberOfPersonVisiting      0
PreferredPropertyStar       0
MaritalStatus               0
NumberOfTrips               0
Passport                    0
OwnCar                      0
NumberOfChildrenVisiting    0
Designation                 0
MonthlyIncome               0
dtype: int64

Let's create dummy variables for string type variables and convert other column types back to float.

In [ ]:

            
                Copied!
                
#converting data types of columns to float
for column in ['NumberOfPersonVisiting', 'Passport', 'OwnCar']:
    X_train[column]=X_train[column].astype('float')
    X_test[column]=X_test[column].astype('float')
#converting data types of columns to float
for column in ['NumberOfPersonVisiting', 'Passport', 'OwnCar']:
    X_train[column]=X_train[column].astype('float')
    X_test[column]=X_test[column].astype('float')

In [ ]:

            
                Copied!
                
#List of columns to create a dummy variables
col_dummy=['TypeofContact', 'Occupation', 'Gender', 'MaritalStatus', 'Designation', 'CityTier']
#List of columns to create a dummy variables
col_dummy=['TypeofContact', 'Occupation', 'Gender', 'MaritalStatus', 'Designation', 'CityTier']

In [ ]:

            
                Copied!
                
#Encoding categorical varaibles
X_train=pd.get_dummies(X_train, columns=col_dummy, drop_first=True)
X_test=pd.get_dummies(X_test, columns=col_dummy, drop_first=True)
#Encoding categorical varaibles
X_train=pd.get_dummies(X_train, columns=col_dummy, drop_first=True)
X_test=pd.get_dummies(X_test, columns=col_dummy, drop_first=True)

Model evaluation criterion:¶

The model can make wrong predictions as:

Predicting a customer will buy the product and the customer doesn't buy - Loss of resources
Predicting a customer will not buy the product and the customer buys - Loss of opportunity

Which case is more important?

Predicting that customer will not buy the product but he buys i.e. losing on a potential source of income for the company because that customer will not be targeted by the marketing team when he should be targeted.

How to reduce this loss i.e need to reduce False Negatives?
The company wants Recall to be maximized, the greater the Recall lesser the chances of false negatives.

Building the model: Logistic Regression¶

Evaluation Metrics¶

Most metrics derive from the confusion matrix comparing predictions vs. ground truth.

Confusion Matrix¶

A 2×2 table summarizing hits and errors: $$ \begin{array}{c|cc} & \text{Pred. 0} & \text{Pred. 1} \\ \hline \text{True 0} & TN & FP \\ \text{True 1} & FN & TP \\ \end{array} $$

TP: true positives (predicted 1 when true 1)
TN: true negatives (predicted 0 when true 0)
FP: false positives (predicted 1 when true 0) — false alarms
FN: false negatives (predicted 0 when true 1) — misses

Recall¶

Proportion of actual positives we correctly capture: $$ \mathrm{Recall}=\frac{TP}{TP+FN}. $$ Higher recall means fewer misses (FN). Choose when the cost of FN is high.

Precision¶

Proportion of predicted positives that are actually positive: $$ \mathrm{Precision}=\frac{TP}{TP+FP}. $$ Higher precision means fewer false alarms (FP). Choose when the cost of FP is high.

In [ ]:

            
                Copied!
                
                    
                    
                
                

        
# Creating metric function
def metrics_score(actual, predicted):
    print(classification_report(actual, predicted))

    cm = confusion_matrix(actual, predicted)
    plt.figure(figsize=(8,5))

    sns.heatmap(cm, annot=True,  fmt='.2f')
    plt.ylabel('Actual')
    plt.xlabel('Predicted')
    plt.show()
# Creating metric function
def metrics_score(actual, predicted):
    print(classification_report(actual, predicted))

    cm = confusion_matrix(actual, predicted)
    plt.figure(figsize=(8,5))

    sns.heatmap(cm, annot=True,  fmt='.2f')
    plt.ylabel('Actual')
    plt.xlabel('Predicted')
    plt.show()

Build a Logistic Regression model (Use the sklearn library) (1 Mark)¶

In [ ]:

            
                Copied!
                
# Fitting logistic regression model
lg = LogisticRegression()
lg.fit(X_train,y_train)
# Fitting logistic regression model
lg = LogisticRegression()
lg.fit(X_train,y_train)

Out[ ]:

LogisticRegression()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Check the performance of the model on train and test data¶

In [ ]:

            
                Copied!
                
# Checking the performance on the training data
y_pred_train = lg.predict(X_train)
metrics_score(y_train, y_pred_train)
# Checking the performance on the training data
y_pred_train = lg.predict(X_train)
metrics_score(y_train, y_pred_train)

              precision    recall  f1-score   support

           0       0.85      0.98      0.91      2777
           1       0.73      0.25      0.37       644

    accuracy                           0.84      3421
   macro avg       0.79      0.61      0.64      3421
weighted avg       0.83      0.84      0.81      3421

Observations:

We have been able to build a predictive model that can be used by the tourist company to predict the customers who are likely to accept the new package with a recall score of 25%.

Let's check the performance on the test set¶

In [ ]:

            
                Copied!
                
# Checking the performance on the test dataset
y_pred_test = lg.predict(X_test)
metrics_score(y_test, y_pred_test)
# Checking the performance on the test dataset
y_pred_test = lg.predict(X_test)
metrics_score(y_test, y_pred_test)

              precision    recall  f1-score   support

           0       0.85      0.98      0.91      1191
           1       0.69      0.23      0.34       276

    accuracy                           0.84      1467
   macro avg       0.77      0.60      0.62      1467
weighted avg       0.82      0.84      0.80      1467

Recap:

Using the model with default threshold the model gives a low recall but decent precision score.
We can’t have both precision and recall high. If you increase precision, it will reduce recall, and vice versa. This is called the precision/recall tradeoff.
So let's find an optimal threshold where we can balance both the metrics.

Find the optimal threshold for the model using the Precision-Recall Curve¶

Precision-Recall curve summarizes the trade-off between the true positive rate and the positive predictive value for a predictive model using different probability thresholds.

Let's use the Precision-Recall curve and see if we can find a better threshold.

In [ ]:

            
                Copied!
                
                    
                    
                
                

        
# Predict_proba gives the probability of each observation belonging to each class
y_scores_lg=lg.predict_proba(X_train)

precisions_lg, recalls_lg, thresholds_lg = precision_recall_curve(y_train, y_scores_lg[:,1])

# Plot values of precisions, recalls, and thresholds
plt.figure(figsize=(10,7))
plt.plot(thresholds_lg, precisions_lg[:-1], 'b--', label='precision')
plt.plot(thresholds_lg, recalls_lg[:-1], 'g--', label = 'recall')
plt.xlabel('Threshold')
plt.legend(loc='upper left')
plt.ylim([0,1])
plt.show()
# Predict_proba gives the probability of each observation belonging to each class
y_scores_lg=lg.predict_proba(X_train)

precisions_lg, recalls_lg, thresholds_lg = precision_recall_curve(y_train, y_scores_lg[:,1])

# Plot values of precisions, recalls, and thresholds
plt.figure(figsize=(10,7))
plt.plot(thresholds_lg, precisions_lg[:-1], 'b--', label='precision')
plt.plot(thresholds_lg, recalls_lg[:-1], 'g--', label = 'recall')
plt.xlabel('Threshold')
plt.legend(loc='upper left')
plt.ylim([0,1])
plt.show()

We want to choose a threshold that has a high recall while also having a small drop in precision. High recall is necessary, simultaneously we also need to be careful not to lose precision too much. So the threshold value of 0.25 should be sufficient because it has good recall and does not cause a significant drop in precision.

Note: We are attempting to maximise recall because that is our metric of interest. Consider the F1 score as the metric of interest then we must find the threshold that provides balanced precision and recall values. In that case, the theshold value will be 0.30.

In [ ]:

            
                Copied!
                
# Setting the optimal threshold
optimal_threshold = 0.25
# Setting the optimal threshold
optimal_threshold = 0.25

Check the performance of the model on train and test data using the optimal threshold. (2 Marks)¶

In [ ]:

            
                Copied!
                
# creating confusion matrix
y_pred_train = lg.predict_proba(X_train)
metrics_score(y_train, y_pred_train[:,1]>optimal_threshold)
# creating confusion matrix
y_pred_train = lg.predict_proba(X_train)
metrics_score(y_train, y_pred_train[:,1]>optimal_threshold)

              precision    recall  f1-score   support

           0       0.90      0.81      0.85      2777
           1       0.42      0.60      0.49       644

    accuracy                           0.77      3421
   macro avg       0.66      0.70      0.67      3421
weighted avg       0.81      0.77      0.78      3421

Recap:

The model performance has improved as compared to our initial model.The recall has increased by 36%.

Let's check the performance on the test set¶

In [ ]:

            
                Copied!
                
y_pred_test = lg.predict_proba(X_test)
metrics_score(y_test, y_pred_test[:,1]>optimal_threshold)
y_pred_test = lg.predict_proba(X_test)
metrics_score(y_test, y_pred_test[:,1]>optimal_threshold)

              precision    recall  f1-score   support

           0       0.91      0.81      0.86      1191
           1       0.45      0.65      0.53       276

    accuracy                           0.78      1467
   macro avg       0.68      0.73      0.70      1467
weighted avg       0.82      0.78      0.80      1467

Recap:

Using the model with a threshold of 0.25, the model has achieved a recall of 67% i.e. increase of 44%.
The precision has dropped compared to inital model but using optimial threshold the model is able to provide the balanced performance.

However the model performance is not good. So let's try building another model.

Logistic Regression and Error Metrics¶

📚 Contents¶

Logistic Regression (theory)¶

Classification and Hypothesis Testing Practice Project: Travel Package Purchase Prediction¶

Context¶

Objective¶

Data Description¶

Importing the libraries required¶

Model Training¶

Loading the dataset¶

Overview of the dataset¶

View the first and last 5 rows of the dataset¶

Understand the shape of the dataset¶

Check the data types of the columns for the dataset¶

Check the percentage of missing values in each column¶

Check the number of unique values in each column¶

Question 1: Check the summary statistics of the dataset and write your observations (2 Marks)¶

Check the count of each unique category in each of the categorical variables.¶

Exploratory Data Analysis¶

Question 2: Univariate Analysis¶

# Question 2.1: Plot the histogram and box plot for the variable Age using the hist_box function provided and write your insights. (1 Mark)¶

# Question 2.2: Plot the histogram and box plot for the variable Duration of Pitch using the hist_box function provided and write your insights. (1 Mark)¶

# Let's understand the distribution of the categorical variables¶

Question 3: Bivariate Analysis¶

# Question 3.1: Find and visualize the correlation matrix using a heatmap and write your observations from the plot. (2 Marks)¶

# Question 3.2: Plot the stacked barplot for the variable Marital Status against the target variable ProdTaken using the stacked_barplot function provided and write your insights. (1 Mark)¶

# Question 3.3: Plot the stacked barplot for the variable ProductPitched against the target variable ProdTaken using the stacked_barplot function provided and write your insights. (1 Mark)¶

Data Preparation for Modeling¶

Model evaluation criterion:¶

Building the model: Logistic Regression¶

Evaluation Metrics¶

Confusion Matrix¶

Recall¶

Precision¶

Build a Logistic Regression model (Use the sklearn library) (1 Mark)¶

Check the performance of the model on train and test data¶

Let's check the performance on the test set¶

Find the optimal threshold for the model using the Precision-Recall Curve¶

Check the performance of the model on train and test data using the optimal threshold. (2 Marks)¶

Let's check the performance on the test set¶

# Question 2.1: Plot the histogram and box plot for the variable `Age` using the hist_box function provided and write your insights. (1 Mark)¶

# Question 2.2: Plot the histogram and box plot for the variable `Duration of Pitch` using the hist_box function provided and write your insights. (1 Mark)¶

# Question 3.2: Plot the stacked barplot for the variable `Marital Status` against the target variable `ProdTaken` using the stacked_barplot function provided and write your insights. (1 Mark)¶

# Question 3.3: Plot the stacked barplot for the variable `ProductPitched` against the target variable `ProdTaken` using the stacked_barplot function provided and write your insights. (1 Mark)¶