Journal
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
Volume 16, Issue 1, Pages 1-17Publisher
IEEE COMPUTER SOC
DOI: 10.1109/TLT.2022.3216535
Keywords
Visualization; Navigation; Feature extraction; Semantics; Data mining; Deep learning; Layout; AutoNote generation; educational videos; hierarchical relationship extraction; video navigation; visual entity segmentation
Ask authors/readers for more resources
With the lack of elaborating annotations and interesting content in educational videos, this article proposes a slide-based video navigation tool that extracts the hierarchical structure and semantic relationship of visual entities in videos by integrating multichannel information. Through a novel deep learning framework, features of visual entities are extracted from presentation slides, and a clustering approach is used to determine the hierarchical relationships between these entities. By evaluating their semantic relationships, visual entities are associated with their corresponding audio speech text, generating a multilevel table of contents and notes for improved learning navigation.
With the increasing popularity of open educational resources in the past few decades, more and more users watch online videos to gain knowledge. However, most educational videos only provide monotonous navigation tools and lack elaborating annotations. This makes the task of locating interesting contents time consuming. To address this limitation, in this article, we propose a slide-based video navigation tool that is able to extract the hierarchical structure and semantic relationship of visual entities in videos, by integrating multichannel information. Features of visual entities are first extracted from the presentation slides by a novel deep learning framework. Then, we propose a clustering approach to extract hierarchical relationships between visual entities (e.g., formulas, texts, or graphs appearing in educational slides). We use this information to associate visual entities with their corresponding audio speech text, by evaluating their semantic relationship. We present two cases where we use the structured data produced by this tool to generate a multilevel table of contents and notes to provide additional navigation materials for learning. The evaluation experiments demonstrate the effectiveness of our proposed solutions for visual entity extraction, hierarchical relationship extraction, as well as corresponding speech text matching. The user study also shows promising improvement in the autogenerated table of contents and notes for facilitating learning.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available