SmoothMV: Seamless Content Adaptation through Head Tracking Analysis and View Prediction

Multi-view has the potential to offer immersive viewing experiences to users, as an alternative to 360 degrees and Virtual Reality (VR) applications. In multi-view, a limited number of camera views are sent to the client and missing views are synthesised locally. Given the substantial complexity associated to view synthesis, considerable attention has been given to optimise the trade-off between bandwidth gains and computing resources, targeting smooth navigation and viewing quality. A still relatively unexplored field is the optimisation of the way navigation interactivity is achieved, i.e. how the user indicates to the system the selection of new viewpoints. In this article, we introduce SmoothMV, a multi-view system that uses a non-intrusive head tracking approach to enhance navigation and Quality of Experience (QoE) of the viewer. It relies on a novel Hot&Cold matrix concept to translate head positioning data into viewing angle selections. Streaming of selected views is done using MPEG-DASH, where a proposed extension to the standard descriptors enables to achieve consistent and flexible view identification.

Automated summary

Research has focused on the complexity and trade-offs of view synthesis to achieve smooth navigation and viewing quality; a novel non-intrusive head tracking approach is used to improve navigation and Quality of Experience (QoE) in a multi-view system.