Intelligent Video Highlights Generation with Front-Camera Emotion Sensing

In this paper, we present HOMER, a cloud-based system for video highlight generation which enables the automated, relevant, and flexible segmentation of videos. Our system outperforms state-of-the-art solutions by fusing internal video content-based features with the user's emotion data. While current research mainly focuses on creating video summaries without the use of affective data, our solution achieves the subjective task of detecting highlights by leveraging human emotions. In two separate experiments, including videos filmed with a dual camera setup, and home videos randomly picked from Microsoft's Video Titles in the Wild (VTW) dataset, HOMER demonstrates an improvement of up to 38% in F-1-score from baseline, while not requiring any external hardware. We demonstrated both the portability and scalability of HOMER through the implementation of two smartphone applications.

Automated summary

HOMER is a cloud-based system for video highlight generation that surpasses state-of-the-art solutions by combining video content features and user emotion data. Through two experiments, it demonstrated an improvement of up to 38% in F-1 score compared to the baseline, without requiring external hardware. The system's portability and scalability were showcased through the implementation of two smartphone applications.