Object-based fusion for urban tree species classification from hyperspectral, panchromatic and nDSM data

This study aims at identifying the best object-based fusion strategy that takes advantage of the complementarity of several heterogeneous airborne data sources for improving the classification of 15 tree species in an urban area (Toulouse, France). The airborne data sources are: hyperspectral Visible Near-Infrared (160 spectral bands, spatial resolution of 0.4 m) and Short-Wavelength Infrared (256 spectral bands, 1.6 m), panchromatic (14 cm), and a normalized Digital Surface Model (12.5 cm). Object-based feature and decision level fusion strategies are proposed and compared when applied to a reference site where the species are previously identified during ground truth collection. This allows the best fusion strategy to be selected with a view to introducing the method in an automatic process (tree crown delineation and species classification) on a test site, independent of the reference site used for learning. In particular, a decision level fusion is selected: based on the Support Vector Machine algorithm, Visible Near-Infrared and Short-Wavelength Infrared classifications use Minimum Noise Fraction components at the original spatial resolution, whereas panchromatic and normalized Digital Surface Model classifications use, respectively, Haralick's and structural features computed at the object scale. After the computation of a decision profile for each source at the object level based on the classification algorithms' membership probabilities, these decision profiles are combined and a decision rule is applied to predict the species. Focusing on the reference site, the Visible Near-Infrared exhibits the best performances with F-score values higher than 60% for 13 species out of 15. The Short-Wavelength Infrared is the most powerful for three species with F-score greater than 60% for seven common species with the Visible Near-Infrared. The panchromatic and normalized Digital Surface Model contribute marginally. The best fusion strategy (decision fusion) does not improve significantly the overall accuracy with 77% (kappa = 74%) against 75% (kappa = 72%) for the Visible Near-Infrared but in general, it improves the results for cases where complementarities have been observed. When applied to the test site and assessed for the two majority species (Tilia tomentosa and Platanus x hispanica), the selected approach gives consistent results with an overall accuracy of 63% against 55% for the Visible Near-Infrared.