Explaining Model Behavior Across Space and Time: Differential and Intertemporal Explanations

Abstract: We introduce the notion of differential explanation as information provided to users of machine learning models that explains why a model assigns different values to two observations. We propose a model-agnostic feature importance method based on SHapley Additive exPlanations (Lundberg and Lee 2017), to generate such explanations that can be viewed as its generalization. We demonstrate the applicability of the method in a wide range of prediction problems that arise in the areas of Information Systems, Operations Management, Finance, and Marketing. We prove that the method is coherent and that in the case of linear models it generates correct answers. Finally, we extend the method to explain changes in predictions over time in dynamic settings, by introducing the concept of intertemporal explanations, which we apply in the context of waiting time prediction.