期刊
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
卷 45, 期 10, 页码 11642-11653出版社
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2023.3278287
关键词
Intensity histogram loss functions; mutual information loss; histogram layers; earth movers distance; image-to-image translation
This paper introduces a novel deep learning framework called HueNet for differentiable construction of intensity and joint histograms. It demonstrates the applicability of HueNet to paired and unpaired image-to-image translation problems by augmenting a generative neural network with histogram layers. The paper also defines two new histogram-based loss functions for constraining the structural appearance and color distribution of the synthesized output image.
We introduce the HueNet a novel deep learning framework for a differentiable construction of intensity (1D) and joint (2D) histograms and present its applicability to paired and unpaired image-to-image translation problems. The key idea is an innovative technique for augmenting a generative neural network by histogram layers appended to the image generator. These histogram layers allow us to define two new histogram-based loss functions for constraining the structural appearance of the synthesized output image and its color distribution. Specifically, the color similarity loss is defined by the Earth Mover's Distance between the intensity histograms of the network output and a color reference image. The structural similarity loss is determined by the mutual information between the output and a content reference image based on their joint histogram. Although the HueNet can be applied to a variety of image-to-image translation problems, we chose to demonstrate its strength on the tasks of color transfer, exemplar-based image colorization, and edges ? photo, where the colors of the output image are predefined. The code is available at https://github.com/mor-avi-aharon-bgu/HueNet.git.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据