Enhanced Motion Compensation for Deep Video Compression

Most of the existing deep learning-based video compression frameworks rely on motion estimation and compensation. However, the artifacts of the warped frames after motion compensation, which propagate the errors to the next frame, limit the video coding performance. In this work, we propose enhanced motion compensation for reduced error propagation in deep video compression. More specifically, we incorporate the designed convolutional neural network into Open DVC as the motion compensation enhancement network to remove noise in the predicted frame. With the enhanced frame, we jointly optimize the whole framework with a single loss function by considering the trade-off between bit cost and frame quality. Experiments show that the proposed enhanced motion compensation model reduces error propagation within a group of frames. Compared with Open DVC, our model can achieve 8.94% bit savings on average for standard test videos in terms of PSNR. Regarding MS-SSIM, our model outperforms Open DVC with 5.67% bit rate savings.

Automated summary

Most deep learning-based video compression frameworks rely on motion estimation and compensation, but the artifacts of warped frames limit the performance. In this work, we propose enhanced motion compensation to reduce error propagation. We incorporate a designed convolutional neural network into Open DVC as the enhancement network, and optimize the framework with a single loss function considering the trade-off between bit cost and frame quality. Experimental results show that our model achieves significant bit savings and outperforms Open DVC in terms of PSNR and bit rate savings.