4.8 Article

MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement

出版社

IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2019.2941941

关键词

Interpolation; Kernel; Estimation; Motion estimation; Adaptation models; Optical imaging; Motion compensation; Motion estimation; motion compensation; convolutional neural network; adaptive warping

资金

  1. National Natural Science Foundation of China [61771306]
  2. Natural Science Foundation of Shanghai [18ZR1418100]
  3. Chinese National Key S&T Special Program [2013ZX01033001-002-002]
  4. Shanghai Key Laboratory of Digital Media Processing and Transmissions [STCSM 18DZ2270700]
  5. NSF Career Grant [1149783]

向作者/读者索取更多资源

The study introduces a motion estimation and compensation driven neural network for video frame interpolation, which integrates optical flow and interpolation kernels using an adaptive warping layer. It achieves visually appealing results without the need for hand-crafted features, showing improved computational efficiency compared to existing methods. The proposed MEMC-Net architecture can be seamlessly adapted to various video enhancement tasks and outperforms state-of-the-art algorithms on a wide range of datasets in quantitative and qualitative evaluations.
Motion estimation (ME) and motion compensation (MC) have been widely used for classical video frame interpolation systems over the past decades. Recently, a number of data-driven frame interpolation methods based on convolutional neural networks have been proposed. However, existing learning based methods typically estimate either flow or compensation kernels, thereby limiting performance on both computational efficiency and interpolation accuracy. In this work, we propose a motion estimation and compensation driven neural network for video frame interpolation. A novel adaptive warping layer is developed to integrate both optical flow and interpolation kernels to synthesize target frame pixels. This layer is fully differentiable such that both the flow and kernel estimation networks can be optimized jointly. The proposed model benefits from the advantages of motion estimation and compensation methods without using hand-crafted features. Compared to existing methods, our approach is computationally efficient and able to generate more visually appealing results. Furthermore, the proposed MEMC-Net architecture can be seamlessly adapted to several video enhancement tasks, e.g., super-resolution, denoising, and deblocking. Extensive quantitative and qualitative evaluations demonstrate that the proposed method performs favorably against the state-of-the-art video frame interpolation and enhancement algorithms on a wide range of datasets.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.8
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据