期刊
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
卷 43, 期 7, 页码 2360-2372出版社
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2020.2969421
关键词
Image segmentation; Semantics; Training; Task analysis; Gallium nitride; Adaptation models; Data models; Domain adaptation; image stylization; semantic segmentation; object detection
The paper presents a simple yet effective domain adaptation framework that aims to reduce the domain gap between synthetic and real data by matching the covariance of universal feature embeddings. By proposing a conditional covariance matching framework, domains can be aligned more precisely, leading to mutual refinement and achieving state-of-the-art domain adaptation results.
Generating computer graphics (CG) rendered synthetic images has been widely used to create simulation environments for robotics/autonomous driving and generate labeled data. Yet, the problem of training models purely with synthetic data remains challenging due to the considerable domain gaps caused by current limitations on rendering. In this paper, we propose a simple yet effective domain adaptation framework towards closing such gap at image level. Unlike many GAN-based approaches, our method aims to match the covariance of the universal feature embeddings across domains, making the adaptation a fast, convenient step and avoiding the need for potentially difficult GAN training. To align domains more precisely, we further propose a conditional covariance matching framework which iteratively estimates semantic segmentation regions and conditionally matches the class-wise feature covariance given the segmentation regions. We demonstrate that both tasks can mutually refine and considerably improve each other, leading to state-of-the-art domain adaptation results. Extensive experiments under multiple synthetic-to-real settings show that our approach exceeds the performance of latest domain adaptation approaches. In addition, we offer a quantitative analysis where our framework shows considerable reduction in Frechet Inception distance between source and target domains, demonstrating the effectiveness of this work in bridging the synthetic-to-real domain gap.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据