Interpreting Machine Learning Models in Python with SHAP
Generic SHAP plot image
Introduction
In machine learning, understanding how models arrive at their predictions is crucial. A common way to determine feature contribution is by looking at feature importance. This measure is based on the decrease in model performance when removing a feature. It is a useful measure but contains no information beyond that importance. SHAP (SHapley Additive exPlanations) is a more impactful measure for explaining the magnitude of feature attributions.
Understanding SHAP Value: Mathematical Perspective
SHAP values are rooted in cooperative game theory, providing a framework to quantify the contribution of each feature to a model's prediction. Mathematically, the SHAP value for a feature can be expressed as follows:
Here’s is an explanation of most of the components in the formula:
In essence, the formula calculates the average contribution of a feature j over all possible combinations of features. It examines the model prediction with and without feature j across all possible feature subsets.
Let’s see an example of implementing and interpreting SHAP values in Python.
Files & Environment
We will use the famous iris dataset that can be accessed using the sklearn-models module.
You can download the Python notebook and YAML file to create the environment in my GitHub repository: GitHub SolisAnalytics.
Importing Packages
We will start by importing the packages and modules.
Evaluating Models with SHAP
Next, we will create a function that trains specified models, evaluates their accuracy, computes SHAP values, and prepares the data for further analysis.
We added the SHAP Tree Explainer for a tree-based model we will use (Random Forest) since it is optimized for such models. The goal is to return SHAP value results and the features in the test set to explain our model’s most interpretable features.
Now, we can load our dataset, specify our models, and run our evaluation function.
Model SHAP values our saved in the results variable.
Interpreting the Results
The SHAP summary plots provide crucial insights into feature impacts on our model. We can focus on Class One to see the most important features in our models.
The slicing operation [:, :, 1] means “give us the SHAP values for all samples (first dimension) for all features (second dimension) but only for the second class.
Let us look at the summary plots to interpret our results. Please remember that we are evaluating SHAP values without data preparation, transformation, and model tuning. These steps can impact feature contributions in a model.
Random Forest Plot Interpretation:
Feature 0 and Feature 2: These features appear to have a mix of positive and negative SHAP values with varying magnitudes, indicating that they have a diverse impact on the model output. For some instances, they increase the likelihood of the target class (positive SHAP values), while for others, they decrease it (negative SHAP values).
Feature 1: This feature mostly has negative SHAP values, suggesting that as the value of Feature 1 increases, the probability of the instance being classified as Class 1 decreases.
Feature 3: This feature shows a cluster of positive SHAP values, indicating that higher values of Feature 3 tend to increase the probability of the instance being classified as Class 1.
The color scale from blue to pink/red shows the low to high values of the features. Points further to the right on the x-axis increase the likelihood of the target being Class 1.
Logistic Regression Plot Interpretation:
Feature 0: Similar to the Random Forest, this feature has a mixture of positive and negative impacts. However, the spread and impact seem larger, suggesting a stronger influence on the prediction for Class 1 in the Logistic Regression model.
Feature 1: This feature predominantly impacts the prediction negatively (more points are on the left side), indicating that higher values of Feature 1 are likely to lower the probability of an instance belonging to Class 1.
Feature 2: This feature has mostly positive SHAP values, implying that it contributes positively to classifying instances as Class 1.
Feature 3: The SHAP values for this feature are also mostly positive but show some spread, suggesting that while generally increasing the likelihood of Class 1, the impact varies across instances.
The beeswarm plot for logistic regression shows a more distinct separation of SHAP values, characteristic of linear models where each feature has a more consistent effect across predictions.
Conclusion
Leveraging SHAP values with a class-specific approach offers valuable insights into determining the main feature contributors in machine learning models. By understanding the distinct contributions of features to different classes, we can enhance our model's interpretability, leading to more informed decisions in the machine learning pipeline. This level of understanding is essential in building transparent, reliable, and effective machine learning solutions.
