Approximating XGBoost with an interpretable decision tree

The increasing usage of machine-learning models in critical domains has recently stressed the necessity of interpretable machine-learning models. In areas like healthcare, finary - the model consumer must understand the rationale behind the model output in order to use it when making a decision. For this reason, it is impossible to use black-box models in these scenarios, regardless of their high predictive performance. Decision forests, and in particular Gradient Boosting Decision Trees (GBDT), are examples of this kind of model. GBDT models are considered the state-of-the-art in many classification challenges, reflected by the fact that the majority of Kaggle's recent winners used GBDT methods as a part of their solution (such as XGBoost). But despite their superior predictive perfor-mance, they cannot be used in tasks that require transparency. This paper presents a novel method for transforming a decision forest of any kind into an interpretable decision tree. The method extends the tool-set available for machine learning practitioners, who want to exploit the interpretability of decision trees without significantly impairing the predic-tive performance gained by GBDT models like XGBoost. We show in an empirical evalua-tion that in some cases the generated tree is able to approximate the predictive performance of a XGBoost model while enabling better transparency of the outputs. (c) 2021 Elsevier Inc. All rights reserved.

Automated summary

The increasing use of machine-learning models in critical domains has highlighted the importance of interpretable machine-learning models. Decision forests, especially Gradient Boosting Decision Trees (GBDT), are considered state-of-the-art in many classification challenges. This paper introduces a novel method for transforming any decision forest into an interpretable decision tree, providing transparency without compromising predictive performance like XGBoost.