4.5 Article

Imitation Learning-Based Algorithm for Drone Cinematography System

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TCDS.2020.3043441

关键词

Cameras; Skeleton; Drones; Visualization; Two dimensional displays; Cinematography; Encoding; Cinematography system; imitation filming; unmanned aerial vehicles; viewpoint control

资金

  1. Natural Science Foundation of China [U1909203]
  2. Zhejiang Province Key Research and Development [2019C01007]
  3. Fundamental Research Funds for the Provincial Universities of Zhejiang [RF-C2019001]

向作者/读者索取更多资源

This study proposes an integrated aerial filming system that autonomously captures cinematic shots of action scenes by imitating demonstrations. The system utilizes the deep deterministic policy gradient to build a model and designs a spatial attention network to selectively focus on discriminative joints of the skeleton. Experimental results demonstrate that our method successfully mimics viewpoint selection strategy and captures more accurate viewpoints compared to existing techniques.
Viewpoint selection for capturing human motion is an important task in autonomous aerial videography, animation, and virtual 3-D environments. Existing methods rely on heuristics for selecting the best viewpoint, which requires human effort to summarize and integrate viewpoint selection rules into a visual servo system to control a camera. In this work, we propose an integrated aerial filming system for autonomously capturing cinematic shots of action scenes on the basis of a set of demonstrations given for imitation. Our model, which is built on the basis of the deep deterministic policy gradient, takes a sequence of a subject's skeleton and the camera pose as input and outputs the camera motion with an optimal viewpoint related to the subject. In addition, we design a spatial attention network to selectively focus on the discriminative joints of the skeleton within each frame. Given the demonstrations with human motions, our framework learns to predict the next best viewpoint by imitating the demonstrations for viewing the motion of the subject. Extensive experimental results in simulated and real outdoor environments demonstrate that our method can successfully mimic the viewpoint selection strategy and capture a more accurate viewpoint than state-of-the-art autonomous cinematography methods.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.5
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据