期刊
EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING
卷 2023, 期 1, 页码 -出版社
SPRINGER
DOI: 10.1186/s13634-023-01008-z
关键词
Data fusion; Out-of-distribution; Missing modality; Robustness; Remote sensing
This paper proposes a method to incorporate source-wise out-of-distribution (OOD) detection into the fusion process of SAR and optical satellite data, aiming to improve the robustness to different types of OOD data and maintain the classification performance. The method adjusts the weights of extracted information based on the in-distribution probabilities, and shows significant improvement in handling individual data source failures or cloud coverage.
The fusion of synthetic aperture radar (SAR) and optical satellite data is widely used for deep learning based scene classification. Counter-intuitively such neural networks are still sensitive to changes in single data sources, which can lead to unexpected behavior and a significant drop in performance when individual sensors fail or when clouds obscure the optical image. In this paper we incorporate source-wise out-of-distribution (OOD) detection into the fusion process at test time in order to not consider unuseful or even harmful information for the prediction. As a result, we propose a modified training procedure together with an adaptive fusion approach that weights the extracted information based on the source-wise in-distribution probabilities. We evaluate the proposed approach on the BigEarthNet multilabel scene classification data set and several additional OOD test cases as missing or damaged data, clouds, unknown classes, and coverage by snow and ice. The results show a significant improvement in robustness to different types of OOD data affecting only individual data sources. At the same time the approach maintains the classification performance of the baseline approaches compared. The code for the experiments of this paper is available on GitHub:
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据