An NLP-guided ontology development and refinement approach to represent and query visual information

The ubiquitous presence of surveillance systems generates massive amounts of video data. Storage and analysis of this data in real-time is a substantial challenge. There is huge potential in representing data in machine-readable and machine-interpretable format due to the presence of hidden semantics in images and videos. However, such representation requires ontology, which calls for expert domain knowledge. In this paper, a novel NLP-guided approach to generate an ontology for multimedia representation and information retrieval is proposed. A semi-automatic NLP-guided framework, which extracts all possible relations among objects is presented. This framework leverages the textual data of the domain to generate possible descriptions and actions within the domain. Relations among objects get embedded as object properties, whereas the category of an object as a class. Features and attributes of objects encode the data properties of the ontology. The proposed ontology is compared with existing multimedia ontologies and evaluated with regard to its capability to represent relations occurring in benchmark datasets, demonstrating the completeness and thorough coverage of the domain concepts. Spatial reasoning rules are established using Semantic Web Rule Language (SWRL) rules, and information retrieval is demonstrated using Description Logic (DL) and SPARQL queries. The proposed NLP-guided ontology generation approach is general enough to help in the development of ontologies for other domains as well, by providing video and textual data of the domain of interest, with limited human involvement.

Automated summary

This paper proposes a NLP-guided approach to generate an ontology for multimedia representation and information retrieval. By leveraging textual data, possible descriptions and actions are generated, and relations among objects are embedded as object properties and classes. The completeness and coverage of the ontology are demonstrated through comparison with existing multimedia ontologies and evaluation. Spatial reasoning rules are established using SWRL rules, and information retrieval is demonstrated using DL and SPARQL queries.