Direct spatio-spectral datacube reconstruction from raw data using a spatially adaptive spatio-spectral basis

Spectral reflectance is an inherent property of objects that is useful for many computer vision tasks. The spectral reflectance of a scene can be described as a spatio-spectral (SS) datacube, in which each value represents the reflectance at a spatial location and a wavelength. In this paper, we propose a novel method that reconstructs the SS datacube from raw data obtained by an image sensor equipped with a multispectral filter array. In our proposed method, we describe the SS datacube as a linear combination of spatially adaptive SS basis vectors. In a previous method, spatially invariant SS basis vectors are used for describing the SS datacube. In contrast, we adaptively generate the SS basis vectors for each spatial location. Then, we reconstruct the SS datacube by estimating the linear coefficients of the spatially adaptive SS basis vectors from the raw data. Experimental results demonstrate that our proposed method can accurately reconstruct the SS datacube compared with the method using spatially invariant SS basis vectors.