Near-optimal Nonlinear Regression Trees

We propose Near-optimal Nonlinear Regression Trees with hyperplane splits (NNRTs) that use a polynomial prediction function in the leaf nodes, which we solve by stochastic gradient methods. On synthetic data, we show experimentally that the algorithm converges to the global optimal. We compare NNRTs, ORT-LH, Multivariate Adaptive Regression Splines (MARS), Random Forests (RF) and XGBoost on 40 real-world datasets and show that overall NNRTs have a performance edge over all other methods. (C) 2021 Published by Elsevier B.V.

Automated summary

Near-optimal Nonlinear Regression Trees with hyperplane splits (NNRTs) use a polynomial prediction function in leaf nodes and are solved by stochastic gradient methods, showing convergence to the global optimal on synthetic data. When compared to other methods like ORT-LH, Multivariate Adaptive Regression Splines (MARS), Random Forests (RF), and XGBoost on 40 real-world datasets, NNRTs demonstrate superior performance overall.