-
-

5.3. Interpretability & Model Inspection#

-

One way to probe the models we build is to test them against the established knowledge of domain experts. In this final section, we’ll explore how to build intuitions about our machine learning model and avoid pitfalls like spurious correlations. These methods for model interpretability increase our trust into models, but they can also serve as an additional level of reproducibility in our research and a valuable research artefact that can be discussed in a publication.

-

This part of the tutorial will also go into some considerations why the feature importance of tree-based methods can serve as a start but often shouldn’t be used as the sole source of truth regarding feature interpretation of our applied research.

-

This section will introduce tools like shap, discuss feature importance, and manual inspection of models.

+
+

5.3. Explainability, Interpretability & Model Inspection#

+

One way to probe the models we build is to test them against the established knowledge of domain experts.

+

In this section, we’ll explore how to build intuitions about our machine learning model and avoid pitfalls like spurious correlations. These methods for model interpretability increase our trust into models, but they can also serve as an additional level of reproducibility in our research and a valuable research artefact that can be discussed in a publication.

+

We will also go into some considerations why the feature importance of tree-based methods can serve as a start but often shouldn’t be used as the sole source of truth regarding feature interpretation of our applied research.

+

Let’s answer some common questions about explainable AI, and ML interpretability first. For one, there’s a question of the difference between explainability and interpretability. You can make the distinction that some models are inherently more interpretable, these methods can include linear models, RuleFit and decision trees. Explainability is often used as a post-hoc method to explain a models behaviour afterwards. Realistically, interpretability and explainability or often used interchangeably.

+

Generally, we can distinguish between different types of explainability:

+
    +

  • Local: Explaining why a prediction was made for a specific data sample.

    • +

  • Global: How a model generally behaves.

    • +
+

Local methods are great for transparency, such as, explaining why a person’s loan was denied. They’re not great for general conclusions about model performance, however, since a single sample cannot be representative of a dataset. Local explainability is often used in publications as a story-telling device to build intuition, but shouldn’t stand alone.

+

Global methods are more appropriate for model inspection. They can explain the general trends in a model, such as the importance of features in the data. These can be used to evaluate a model against the intuition of domain experts and established knowledge. These global methods can also be used to evaluate the model against a test data set.

+

This section will introduce tools like shap and partial dependence plots, discuss feature importance, and manual inspection of neural network models.

First we’ll split the data from the Data notebook and load the model from the Sharing notebook.

@@ -605,7 +614,7 @@

5.3.1. Partial Dependence for Machine Le from matplotlib import pyplot as plt -pd_results = partial_dependence(model, X_train.sample(20), num_features) +pd_results = partial_dependence(model, X_train.sample(5), num_features) print(pd_results.keys()) print(f"Example Values: {pd_results['values'][0]}, Average: {pd_results['average'][0][0].mean(axis=0)}")