Interpretable Perturbator for Variable Selection in near-Infrared Spectral Analysis

A perturbator was developed for variable selection in near-infrared (NIR) spectral analysis based on the perturbation strategy in deep learning for developing interpretation methods. A deep learning predictor was first constructed to predict the targets from the spectra in the training set. Then, taking the output of the predictor as a reference, the perturbator was trained to derive the perturbation-positive (P+) and perturbation-negative (P-) features from the spectra. Therefore, the weight (sigma) of the perturbator layer can be a criterion to evaluate the importance of the variables in the spectra. Ranking the spectral variables by the criterion, the number of the variables used in the quantitative model can be obtained through cross-validation. Three NIR data sets were used to evaluate the proposed method. The root mean squared error was found to be comparable with or superior to that obtained by the commonly used methods. Moreover, the selected spectral variables are interpretable in identifying the key spectral features related to the prediction target. Therefore, the proposed method provides not only an effective tool for optimizing quantitative model, but also an efficient way for explaining spectra of multicomponent samples.

Automated summary

A perturbator based on deep learning was developed for variable selection in near-infrared spectral analysis. The perturbator was trained to derive positive and negative features from the spectra, and the weight of the perturbator layer can be used to evaluate the importance of the variables. Experimental results showed that the proposed method achieved comparable or better performance in terms of root mean squared error, and the selected spectral variables were interpretable in identifying key spectral features related to the prediction target.