Journal
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Volume 44, Issue 12, Pages 9669-9686Publisher
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2021.3129795
Keywords
Training; Lighting; Image reconstruction; Image decomposition; Surface acoustic waves; Image sequences; Annotations; Intrinsic images; reflectance; shading
Funding
- NSFC [U21A20471, U1911401, U1811461, 61802453]
- Guangdong NSF Project [2020B1515120085, 2018B030312002]
- Guangzhou Research Project [201902010037]
- Research Projects of Zhejiang Lab [2019KD0AB03]
- Key-Area Research and Development Program of Guangzhou [202007030004]
Ask authors/readers for more resources
In this paper, an unsupervised framework for image decomposition is proposed, which can effectively learn the decomposition from a single natural image. By promoting the internal self-similarity of the reflectance component, the framework decomposes the image into reflectance and shading, and trains the network using a novel loss function. Experimental results on benchmark datasets demonstrate the superiority of the proposed method.
Recent learning-based intrinsic image decomposition methods have achieved remarkable progress. However, they usually require massive ground truth intrinsic images for supervised learning, which limits their applicability on real-world images since obtaining ground truth intrinsic decomposition for natural images is very challenging. In this paper, we present an unsupervised framework that is able to learn the decomposition effectively from a single natural image by training solely with the image itself. Our approach is built upon the observations that the reflectance of a natural image typically has high internal self-similarity of patches, and a convolutional generation network tends to boost the self-similarity of an image when trained for image reconstruction. Based on the observations, an unsupervised intrinsic decomposition network (UIDNet) consisting of two fully convolutional encoder-decoder sub-networks, i.e., reflectance prediction network (RPN) and shading prediction network (SPN), is devised to decompose an image into reflectance and shading by promoting the internal self-similarity of the reflectance component, in a way that jointly trains RPN and SPN to reproduce the given image. A novel loss function is also designed to make effective the training for intrinsic decomposition. Experimental results on three benchmark real-world datasets demonstrate the superiority of the proposed method.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available