期刊
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS
卷 41, 期 8, 页码 2645-2657出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/JSAC.2023.3288243
关键词
Semantic communication; wireless communication; joint source-channel coding; perceptual similarity; generative adversarial networks; inverse problems; StyleGAN
Recent works have shown that DeepJSCC, a joint source-channel coding scheme using deep neural networks (DNNs), provides promising results for wireless image transmission. However, existing methods mostly focus on the distortion of reconstructed signals rather than human perception. This work proposes two novel JSCC schemes, InverseJSCC and GenerativeJSCC, that leverage deep generative models for improved perceptual quality in extreme conditions. Simulation results demonstrate that both InverseJSCC and GenerativeJSCC outperform DeepJSCC in terms of perceptual quality and distortion.
Recent works have shown that joint source-channel coding (JSCC) schemes using deep neural networks (DNNs), called DeepJSCC, provide promising results in wireless image transmission. However, these methods mostly focus on the distortion of the reconstructed signals with respect to the input image, rather than their perception by humans. However, focusing on traditional distortion metrics alone does not necessarily result in high perceptual quality, especially in extreme physical conditions, such as very low bandwidth compression ratio (BCR) and low signal-to-noise ratio (SNR) regimes. In this work, we propose two novel JSCC schemes that leverage the perceptual quality of deep generative models (DGMs) for wireless image transmission, namely InverseJSCC and GenerativeJSCC. While the former is an inverse problem approach to DeepJSCC, the latter is an end-to-end optimized JSCC scheme. In both, we optimize a weighted sum of mean squared error (MSE) and learned perceptual image patch similarity (LPIPS) losses, which capture more semantic similarities than other distortion metrics. InverseJSCC performs denoising on the distorted reconstructions of a DeepJSCC model by solving an inverse optimization problem using the pre-trained style-based generative adversarial network (StyleGAN). Our simulation results show that InverseJSCC significantly improves the state-of-the-art DeepJSCC in terms of perceptual quality in edge cases. In GenerativeJSCC, we carry out end-to-end training of an encoder and a StyleGAN-based decoder, and show that GenerativeJSCC significantly outperforms DeepJSCC both in terms of distortion and perceptual quality.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据