Universum parametric ν-support vector regression for binary classification problems with its applications

Universum data sets, a collection of data sets that do not belong to any specific class in a classification problem, give previous information about data in the mathematical problem under consideration to enhance the classifiers' generalization performance. Recently, some researchers have integrated Universum data into the existing models to propose new models which result in improved classification performance. Inspired by these Universum models, an efficient parametric nu-support vector regression with Universum data (UPar-nu-SVR) is proposed in this work. This method, which finds two non-parallel hyperplanes by solving one optimization problem and considers heteroscedastic noise, overcomes some common disadvantages of the previous methods. The UPar-nu-SVR includes unlabeled samples that don't belong to any class in the training process, which results in a quadratic programming problem. Two approaches are proposed to solve this problem. The first approach derives the dual formulation using the Lagrangian function and KKT conditions. Furthermore, a least squares parametric nu-support vector regression with Universum data (named LS-UPar-nu-SVR) is suggested to further increase the generalization performance. The LS-UPar-nu-SVR solves a single system of linear equations, instead of addressing a quadratic programming problem in the dual formulation. Numerical experiments on artificial, UCI, credit card, NDC, and handwritten digit recognition data sets show that the suggested Universum model and its solving methodologies improve prediction accuracy.

Automated summary

In this work, an efficient parametric nu-support vector regression model with Universum data (UPar-nu-SVR) is proposed, which overcomes some common disadvantages of previous methods. Two approaches are suggested to solve the quadratic programming problem caused by including unlabeled samples. Furthermore, a least squares parametric nu-support vector regression model with Universum data (LS-UPar-nu-SVR) is also proposed to improve the generalization performance.