Interpretable machine learning with an ensemble of gradient boosting machines

A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using randomized decision trees of depth 1, which provide a very simple architecture. Weights of GBMs as well as features are computed in each iteration of boosting by using the Lasso method and then updated by means of a specific smoothing procedure. In contrast to the neural additive model, the method provides weights of features in the explicit form, and it is simply trained. A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation. (C) 2021 Elsevier B.V. All rights reserved.

Automated summary

This method proposes a way to interpret black-box models locally and globally based on ensemble gradient boosting machines, using simple decision tree structures and the Lasso method for weight calculation and update. Compared to the neural additive model, it provides a more intuitive and easy-to-train approach.