Pre-Processing Training Data Improves Accuracy and Generalisability of Convolutional Neural Network Based Landscape Semantic Segmentation

Data pre-processing for developing a generalised land use and land cover (LULC) deep learning model using earth observation data is important for the classification of a different date and/or sensor. However, it is unclear how to approach deep learning segmentation problems in earth observation data. In this paper, we trialled different methods of data preparation for Convolutional Neural Network (CNN) training and semantic segmentation of LULC features within aerial photography over the Wet Tropics and Atherton Tablelands, Queensland, Australia. This was conducted by trialling and ranking various training patch selection sampling strategies, patch and batch sizes, data augmentations and scaling and inference strategies. Our results showed: a stratified random sampling approach for producing training patches counteracted class imbalances; a smaller number of larger patches (small batch size) improves model accuracy; data augmentations and scaling are imperative in creating a generalised model able to accurately classify LULC features in imagery from a different date and sensor; and producing the output classification by averaging multiple grids of patches and three rotated versions of each patch produced a more accurate and aesthetic result. Combining the findings from the trials, we fully trained five models on the 2018 training image and applied the model to the 2015 test image. The output LULC classifications achieved an average kappa of 0.84, user accuracy of 0.81, and producer accuracy of 0.87. Future research using CNNs and earth observation data should implement the findings of this project to increase LULC model accuracy and transferability.

Automated summary

Data pre-processing is important for developing a generalised land use and land cover (LULC) deep learning model using earth observation data. This paper trialled different methods of data preparation for Convolutional Neural Network (CNN) training and achieved accurate classification of LULC features in aerial photography. The results suggest that stratified random sampling, smaller batch sizes, data augmentations, scaling, and averaging multiple grids of patches improved the model accuracy and aesthetic result.