Deep Exemplar-based Video Colorization

This paper presents the first end-to-end network for exemplar-based video colorization. The main challenge is to achieve temporal consistency while remainingfaithful to the reference style. To address this issue, we introduce a recurrentframework that unifies the semantic correspondence and color propagationsteps. Both steps allow a provided reference image to guide the colorization of every frame, thus reducing accumulatedpropagationerrors. Video frames are colorized in sequence based on the colorization history, and its coherency is further enforced by the temporal consistency loss. All of these components, learnt end-to-end, help produce realistic videos with good temporal stability. Experiments show our result is superior to the state-of-the-artmethods both quantitativelyand qualitatively.