Fully adaptive dictionary for online correntropy kernel learning using proximal methods

We introduce a new sparse variant of the Correntropy Kernel Learning model, hereafter named Fully ADaptive Online Sparse CKL (FADOS-CKL), for online system identification in the presence of outliers. For this purpose, we develop a fully adaptive dictionary of support vectors (SVs) so that it can either grow (as most of the kernel models to date do) or shrink if some of the SVs become obsolete with time. For inclusion of SVs into the dictionary, existing strategies (ALD, Novelty, Surprise, and Coherence) have their performances compared in this paper, while for elimination of SVs we adopt a class of optimization techniques known as proximal methods. Dictionary updating in FADOS-CKL is carried out on-the-fly by the introduction of a recursive methodology based on the Sherman-Morrison-Woodbury formula to update the kernel matrix and its inverse with low computational complexity. Aiming at achieving the smallest predictive errors with the highest sparsity level, a comprehensive performance comparison involving the FADOS-CKL model and powerful alternatives is carried out using two large-scale benchmark datasets for different levels of outliers contamination. The results indicate an impressive balance between reduction in the dictionary size and the corresponding generalization capability of the proposed FADOS-CKL model over the existing alternatives.

Automated summary

The paper introduces a new sparse variant of the Correntropy Kernel Learning model named FADOS-CKL for online system identification in the presence of outliers. With a fully adaptive dictionary of support vectors (SVs) and optimization techniques, FADOS-CKL achieves a balance between reducing dictionary size and maintaining high generalization capability. Performance comparisons with powerful alternatives on large benchmark datasets show the effectiveness of FADOS-CKL in minimizing predictive errors and achieving high sparsity levels.