Robust partially linear models for automatic structure discovery

Partially linear models (PLMs), rooted in the combination of linear and nonlinear approximation, are recognized to be capable of modeling complex data. Indeed, the performance of PLMs depends heavily on the choice of model structure, such as which covariates have linear or nonlinear effects on the response. Nevertheless, most existing PLMs are limited to the mean regression, resulting in sensitivity to non-Gaussian noises, such as skewed noise and heavy-tailed noise. In order to mitigate the influence of noise in structure discovery, this paper proposes a Robust Linear And Nonlinear Discovery algorithm (RLAND) by integrating the modal regression and PLMs. Statistical analysis on generalization bound and structure discovery consistency are established to characterize its learning theory foundations. Computation analysis illustrates that the RLAND can be efficiently realized by half quadratic optimization and the quadratic programming. Empirical evaluations on simulation and real-world data validate the competitive performance of the proposed method.

Automated summary

Partially linear models (PLMs), which combine linear and nonlinear approximation, are effective in modeling complex data. However, most existing PLMs are limited to mean regression and are sensitive to non-Gaussian noises. To address this issue, this paper proposes a Robust Linear And Nonlinear Discovery algorithm (RLAND) that integrates modal regression and PLMs. The algorithm is supported by statistical analysis on generalization bound and structure discovery consistency, and it can be efficiently computed using half quadratic optimization and quadratic programming. Empirical evaluations on simulated and real-world data demonstrate the competitive performance of the proposed method.