期刊
IEEE TRANSACTIONS ON MULTIMEDIA
卷 22, 期 9, 页码 2331-2344出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMM.2019.2957986
关键词
Head; Visualization; Feature extraction; Predictive models; Magnetic heads; Frequency-domain analysis; Uncertainty; VR; 360 degree; saliency; head-eye motion; scanpath; spherical harmonics; center and peripheral vision
资金
- National Natural Science Foundation of China [61831015, 61771305, 61521062, 61527804]
- China Postdoctoral Science Foundation [BX20180197, 2019M651496]
- Macau Science and Technology Development Fund [FDCT/022/2017/A1, FDCT/077/2018/A2]
- Research Committee at the University of Macau [MYRG2016-00137-FST, MYRG2018-00029-FST]
By recording the whole scene around the capturer, virtual reality (VR) techniques can provide viewers the sense of presence. To provide a satisfactory quality of experience, there should be at least 60 pixels per degree, so the resolution of panoramas should reach 21600 x 10800. The huge amount of data will put great demands on data processing and transmission. However, when exploring in the virtual environment, viewers only perceive the content in the current field of view (FOV). Therefore if we can predict the head and eye movements which are important behaviors of viewer, more processing resources can be allocated to the active FOV. But conventional saliency prediction methods are not fully adequate for panoramic images. In this paper, a new panorama-oriented model, to predict head and eye movements, is proposed. Due to the superiority of computation in the spherical domain, the spherical harmonics are employed to extract features at different frequency bands and orientations. Related low- and high-level features including the rare components in the frequency domain and color domain, the difference between center vision and peripheral vision, visual equilibrium, person and car detection, and equator bias are extracted to estimate the saliency. To predict head movements, visual mechanisms including visual uncertainty and equilibrium are incorporated, and the graphical model and functional representation for the switch of head orientation are established. Extensive experimental results on the publicly available database demonstrate the effectiveness of our methods.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据