期刊
ACM TRANSACTIONS ON GRAPHICS
卷 41, 期 6, 页码 -出版社
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3550454.3555471
关键词
colorization; multi-modal controls; color editing; Transformer
资金
- Hong Kong Research Grants Council (RGC) GRF Scheme [CityU 11216122]
This paper proposes a unified framework UniColor to support colorization in multiple modalities. The framework includes a two-stage colorization approach and a Transformer-based network to generate diverse and high-quality colorization results. Both qualitative and quantitative comparisons demonstrate the superiority of the proposed method. An interactive interface is also designed to showcase the effectiveness of the unified framework in practical usage.
We propose the first unified framework UniColor to support colorization in multiple modalities, including both unconditional and conditional ones, such as stroke, exemplar, text, and even a mix of them. Rather than learning a separate model for each type of condition, we introduce a two-stage colorization framework for incorporating various conditions into a single model. In the first stage, multi-modal conditions are converted into a common representation of hint points. Particularly, we propose a novel CLIP-based method to convert the text to hint points. In the second stage, we propose a Transformer-based network composed of Chroma-VQGAN and Hybrid-Transformer to generate diverse and high-quality colorization results conditioned on hint points. Both qualitative and quantitative comparisons demonstrate that our method outperforms state-of-the-art methods in every control modality and further enables multi-modal colorization that was not feasible before. Moreover, we design an interactive interface showing the effectiveness of our unified framework in practical usage, including automatic colorization, hybrid-control colorization, local recolorization, and iterative color editing.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据