期刊
IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS
卷 14, 期 2, 页码 403-413出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TCDS.2020.3043441
关键词
Cameras; Skeleton; Drones; Visualization; Two dimensional displays; Cinematography; Encoding; Cinematography system; imitation filming; unmanned aerial vehicles; viewpoint control
资金
- Natural Science Foundation of China [U1909203]
- Zhejiang Province Key Research and Development [2019C01007]
- Fundamental Research Funds for the Provincial Universities of Zhejiang [RF-C2019001]
This study proposes an integrated aerial filming system that autonomously captures cinematic shots of action scenes by imitating demonstrations. The system utilizes the deep deterministic policy gradient to build a model and designs a spatial attention network to selectively focus on discriminative joints of the skeleton. Experimental results demonstrate that our method successfully mimics viewpoint selection strategy and captures more accurate viewpoints compared to existing techniques.
Viewpoint selection for capturing human motion is an important task in autonomous aerial videography, animation, and virtual 3-D environments. Existing methods rely on heuristics for selecting the best viewpoint, which requires human effort to summarize and integrate viewpoint selection rules into a visual servo system to control a camera. In this work, we propose an integrated aerial filming system for autonomously capturing cinematic shots of action scenes on the basis of a set of demonstrations given for imitation. Our model, which is built on the basis of the deep deterministic policy gradient, takes a sequence of a subject's skeleton and the camera pose as input and outputs the camera motion with an optimal viewpoint related to the subject. In addition, we design a spatial attention network to selectively focus on the discriminative joints of the skeleton within each frame. Given the demonstrations with human motions, our framework learns to predict the next best viewpoint by imitating the demonstrations for viewing the motion of the subject. Extensive experimental results in simulated and real outdoor environments demonstrate that our method can successfully mimic the viewpoint selection strategy and capture a more accurate viewpoint than state-of-the-art autonomous cinematography methods.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据