4.7 Article

Viewpoint Recommendation Based on Object-Oriented 3D Scene Reconstruction

Journal

IEEE TRANSACTIONS ON MULTIMEDIA
Volume 23, Issue -, Pages 257-267

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMM.2020.2981237

Keywords

Three-dimensional displays; Object detection; Cameras; Feature extraction; Image reconstruction; Social networking (online); Two dimensional displays; 3D reconstruction; social media; aesthetics evaluation; viewpoint recommendation

Funding

  1. NSFC [61732008, 61772407]
  2. Guangdong Provincial Science and Technology Plan [2016A010101005]
  3. Microsoft Research Asia

Ask authors/readers for more resources

The paper introduces a system based on social media and 3D reconstruction to recommend good viewpoints for taking high-quality photographs. By using weakly supervised object detection and 3D reconstruction, in combination with aesthetics and diversity, several high-quality viewpoints are recommended.
Viewpoint recommendation can recommend several viewpoints for taking aesthetic photographs of a place-of-interest (POI) and is of great importance for photography assistance. In this paper, we propose a system that can assist a user in choosing good viewpoints for taking high-quality photographs. Our system is based on social media and 3D reconstruction. To reduce the time cost and improve the quality of 3D reconstruction, we propose a weakly supervised object detection method that is used before 3D reconstruction. The camera pose of images is recovered by the subsequent 3D reconstruction pipeline. We use a convolutional neural network (CNN) to extract 2D image features, and we fuse them with 3D camera pose features to learn their relationships to image aesthetics. The trained model is utilized to evaluate the aesthetics of images. Finally, the 3D space of all possible camera poses is divided into 3D grids, and the aesthetics score of each grid is evaluated. We combine the aesthetics and diversity of all viewpoints and recommend several high-quality viewpoints. Experimental results indicate that our approach can help users choose viewpoints that will result in high-quality photographs while maintaining diversity.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available