Appearance based deep domain adaptation for the classification of aerial images

This paper addresses appearance based domain adaptation for the pixel-wise classification of remotely sensed data using deep neural networks (DNN) as a strategy to reduce the requirements of DNN with respect to the availability of training data. We focus on the setting in which labelled data are only available in a source domain D S, but not in a target domain D T, known as unsupervised domain adaptation in Computer Vision. Our method is based on adversarial training of an appearance adaptation network (AAN) that transforms images from D S such that they look like images from D T. Together with the original label maps from D S, the transformed images are used to adapt a DNN to D T. The AAN has to change the appearance of objects of a certain class such that they resemble objects of the same class in D T. Many approaches try to achieve this goal by incorporating cycle consistency in the adaptation process, but such approaches tend to hallucinate structures that occur frequently in one of the domains. In contrast, we propose a joint training strategy of the AAN and the classifier, which constrains the AAN to transform the images such that they are correctly classified. To further improve the adaptation performance, we propose a new regularization loss for the discriminator network used in adversarial training. We also address the problem of finding the optimal values of the trained network parameters, proposing a new unsupervised entropy based parameter selection criterion, which compensates for the fact that there is no validation set in D T that could be monitored. As a minor contribution, we present a new weighting strategy for the cross-entropy loss, addressing the problem of imbalanced class distributions. Our method is evaluated in 42 adaptation scenarios using datasets from 7 cities, all consisting of high-resolution digital orthophotos and height data. It achieves a positive transfer in all cases, and on average it improves the performance in the target domain by 4.3% in overall accuracy. In adaptation scenarios between the Vaihingen and Potsdam datasets from the ISPRS semantic labelling benchmark our method outperforms those from recent publications by 10 20% with respect to the mean intersection over union.

Automated summary

This paper proposes an appearance-based domain adaptation method for pixel-wise classification of remotely sensed data using deep neural networks (DNN). The method utilizes adversarial training of an appearance adaptation network (AAN) and a classifier in a joint training strategy to improve the performance of DNN in scenarios where labeled data is only available in the source domain. By incorporating new regularization loss, parameter selection criterion, and weighting strategy, the method achieves positive transfer in all cases and outperforms recent publications in average intersection over union by 10-20%, demonstrating its efficacy in adaptation scenarios.