Integrating Hyperspectral Likelihoods in a Multidimensional Assignment Algorithm for Aerial Vehicle Tracking

Tracking vehicles through dense environments is an important and challenging task that is mostly tackled using visible and near IR wavelengths. Hyperspectral imaging is known to improve the robustness of target identification, but the massive increase in data created is usually prohibitive for tracking many targets. We present a persistent real-time aerial target tracking system, taking advantage of an adaptive, multimodal sensor concept and blending the hyperspectral likelihoods with kinematic likelihoods in a multidimensional assignment framework. The adaptive sensor is capable of providing wide field of view panchromatic images as well as the spectra of small number of pixels. The proposed system does not require large amount of hyperspectral data collection as we focus on tracking fewer number of targets with higher persistency. This overcomes the data challenge of hyperspectral tracking by following dynamic data-driven application systems (DDDAS) principles to control hyperspectral data collection where most beneficial. The DDDAS framework for controlling hyperspectral data collection is developed by incorporating prior information from the filter movement predictions and information from motion detection. The proposed multidimensional hyperspectral feature-aided tracker is compared to a 2-D hyperspectral feature-aided tracker and another cascaded hyperspectral data based tracker by generating a synthetic, realistic, aerial video on a dense scene.