Estimation of number of spectrally distinct signal sources in hyperspectral imagery

With very high spectral resolution, hyperspectral sensors can now uncover many unknown signal sources which cannot be identified by visual inspection or a priori. In order to account for such unknown signal sources, we introduce a new definition, referred to as virtual dimensionality (VD) in this paper. It is defined as the minimum number of spectrally distinct signal sources that characterize the hyperspectral data from the perspective view of target detection and classification. It is different from the commonly used intrinsic dimensionality (ID) in the sense that the signal sources are determined by the proposed VD based only on their distinct spectral properties. These signal sources may include unknown interfering sources, which cannot be identified by prior knowledge. With this new definition, three Neyman-Pearson detection theory-based thresholding methods are developed to determine the VD of hyperspectral imagery, where eigenvalues are used to measure signal energies in a detection model. In order to evaluate the performance of the proposed methods, two information criteria, an information criterion (AIC) and minimum description length (MDL), and the factor analysis-based method proposed by Malinowski, are considered for comparative analysis. As demonstrated in computer simulations, all the methods and criteria studied in this paper may work effectively when noise is independent identically distributed. This is, unfortunately, not true when some of them are applied to real image data. Experiments show that all the three eigenthresholding based methods (i.e., the Harsanyi-Farrand-Chang (HFC), the noise-whitened HFC (NWHFC), and the noise subspace projection (NSP) methods) produce more reliable estimates of VD compared to the AIC, MDL, and Malinowski's empirical indicator function, which generally overestimate VD significantly. In summary, three contributions are made in this paper, 1) an introduction of the new definition of VD, 2) three Neyman-Pearson detection theory-based thresholding methods, HFC, NWHFC, and NSP derived for VD estimation, and 3) experiments that show the AIC and MDL commonly used in passive array processing and the second-order statistic-based Malinowski's method are not effective measures in VD estimation.