Deep Exemplar-Based Color Transfer for 3D Model

Recoloring 3D models is a challenging task that often requires professional knowledge and tedious manual efforts. In this article, we present the first deep-learning framework for exemplar-based 3D model recolor, which can automatically transfer the colors from a reference image to the 3D model texture. Our framework consists of two modules to solve two major challenges in the 3D color transfer. First, we propose a new feed-forward Color Transfer Network to achieve high-quality semantic-level color transfer by finding dense semantic correspondences between images. Second, considering 3D model constraints such as UV mapping, we design a novel 3D Texture Optimization Module which can generate a seamless and coherent texture by combining color transferred results rendered in multiple views. Experiments show that our method performs robustly and generalizes well to various kinds of models.

Automated summary

In this research, a deep-learning framework for exemplar-based 3D model recoloring is proposed. The framework achieves high-quality semantic-level color transfer by finding semantic correspondences between images, and generates seamless and coherent textures by utilizing the UV mapping of 3D models.