4.5 Article

Video frame interpolation for high dynamic range sequences captured with dual-exposure sensors

Journal

COMPUTER GRAPHICS FORUM
Volume 42, Issue 2, Pages 119-131

Publisher

WILEY
DOI: 10.1111/cgf.14748

Keywords

-

Ask authors/readers for more resources

This study explores the advantages of using dual-exposure sensors to accurately handle HDR scenes with complex motion. By spatially registering sharp short and blurry long exposures and aligning their ends temporally, motion blur provides continuous information on scene motion that, combined with the reference, enables more precise motion sampling. The proposed neural network outperforms existing solutions in VFI tasks and HDR frame reconstruction.
Video frame interpolation (VFI) enables many important applications such as slow motion playback and frame rate conversion. However, one major challenge in using VFI is accurately handling high dynamic range (HDR) scenes with complex motion. To this end, we explore the possible advantages of dual-exposure sensors that readily provide sharp short and blurry long exposures that are spatially registered and whose ends are temporally aligned. This way, motion blur registers temporally continuous information on the scene motion that, combined with the sharp reference, enables more precise motion sampling within a single camera shot. We demonstrate that this facilitates a more complex motion reconstruction in the VFI task, as well as HDR frame reconstruction that so far has been considered only for the originally captured frames, not in-between interpolated frames. We design a neural network trained in these tasks that clearly outperforms existing solutions. We also propose a metric for scene motion complexity that provides important insights into the performance of VFI methods at test time.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.5
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available