Week 7 – Model Evaluation

Introduction

In Week 7, you will enhance your previously developed classification models (from Week 6) by improving their accuracy and generalization ability through hyperparameter tuning, cross-validation, and feature importance analysis. This week is about optimization — turning your baseline models into high-performing, fine-tuned prediction systems for your E-commerce Recommendation System project.

Objectives

• Optimize Logistic Regression and Random Forest models.
• Perform hyperparameter tuning using GridSearchCV or RandomizedSearchCV.
• Evaluate models using cross-validation scores.
• Analyze feature importance to identify key predictors.
• Compare tuned models with baseline models from Week 6.

📝 Load and Preprocess Data

Key Steps in Week 7 Assignment

Use the same cleaned dataset from Week 6. Ensure:

• No missing values.
• Encoded categorical features.
• Standardized numeric features.

You can reuse your preprocessing code from Week 6.

✅ Preprocessing Completed

📊 Baseline Accuracy

Understanding GridSearchCV Output

🔍 Step-by-Step Explanation

When you see messages like the ones above, they come from GridSearchCV in Scikit-Learn, which automatically tests multiple parameter combinations to find the best-performing model.

1️⃣ Fitting 5 folds for each of 6 candidates

This refers to the Logistic Regression model.

• 6 candidates means GridSearchCV is testing 6 possible hyperparameter combinations.
• 5 folds means it’s using 5-fold cross-validation, splitting the data into 5 parts and training/testing 5 times.

That’s why you see: 6 × 5 = 30 fits — meaning 30 training and testing cycles in total.

2️⃣ Fitting 3 folds for each of 8 candidates

This message corresponds to the Random Forest model.

• 8 candidates means 8 different hyperparameter combinations are being tested.
• 3 folds means 3-fold cross-validation is used to balance accuracy with training speed.

Total = 8 × 3 = 24 fits.

🏆 Best Hyperparameters Found

After testing all parameter combinations, GridSearchCV identifies which settings produce the highest validation accuracy.

⚙️ Hyperparameter Explanation

• C = 0.1: Controls regularization strength. Smaller values simplify the model.
• penalty = 'l1': Uses L1 regularization, which helps with feature selection.
• solver = 'liblinear': A fast algorithm for small to medium datasets.
• max_depth = 10: Limits how deep trees can grow to avoid overfitting.
• min_samples_leaf = 2: Ensures at least 2 samples per leaf, improving stability.
• n_estimators = 100: Builds 100 trees for better generalization.

🏆 Best Hyperparameters Found

📈 Tuned Logistic Regression Performance

Accuracy: 0.8264

📈 Tuned Random Forest Performance

Accuracy: 0.8271

🔁 Cross-Validation Accuracy

📊 Interpretation of Feature Importance

📘 Final Interpretation of Charts

• Logistic Regression: The tuned version slightly improves accuracy but remains limited by its linear assumptions. It’s easier to interpret but less flexible.
• Random Forest: Delivers higher accuracy both before and after tuning, demonstrating strong performance on complex data structures. Tuning fine-tunes depth and split rules for optimal generalization.
• Model Comparison Chart: Clearly shows accuracy improvement after tuning. The largest jump is seen in Random Forest, confirming that ensemble methods capture non-linear relationships more effectively.
• Business Takeaway: The tuned Random Forest model is the best-performing approach for predicting customer satisfaction, offering actionable insights into which service factors most influence satisfaction.