Distance and similarity-search metrics for use with soil vis-NIR spectra

Many techniques used in soil vis-NIR sensing are based on the measurement of the similarity or distance between samples. The question that frequently arises when two samples are very close in the vis-NIR space is whether they are also close (or similar) in terms of soil compositional characteristics. A good soil vis-NIR similarity metric must be also able to reflect the soil compositional similarity. In this respect, the main aims of this work were as follows: L investigate the relationship between soil vis-NIR similarity and soil compositional similarity and ii. evaluate different distance metric algorithms for soil vis-NIR similarity search. We evaluated the following distance metrics: Euclidean (ED), Mahalanobis (MD), spectral angle mapper (SAM), surface difference spectrum (SDS), spectral information divergence (SID), principal component distance (PC-M), optimized PC distance (oPC-M), locally linear embedding distance (LLE-M) and sigma-locally linear embedding (sigma LLE-M). The first five methods mentioned previously correspond to methods that operate in the spectral space while the remaining ones work by projecting the vis-NIR data onto a low dimensional space. We used a global soil vis-NIR spectral library (GSSL) to test the different distance metric algorithms. The GSSL was divided into a reference set (Xr) and an unknown set (Xu). The distance algorithms were used to find in Xr the most spectrally similar samples of Xu. In order to evaluate the compositional similarity, the clay content and pH values of the Xu were compared to the clay content and pH values of the samples found in Xr by each algorithm. The experimental results showed that the vis-NIR similarity measures that better reflect the soil compositional similarity are those corresponding to the oPC-M, LLE-M and sigma LLE-M methods. We also show that the SDS approach is a suitable method for computing distances in the spectral space. Finally, in this paper we discuss how these methods can also be used in proximal soil vis-NIR sensing applications. (C) 2012 Elsevier B.V. All rights reserved.