A Survey on Multimodal Knowledge Graphs: Construction, Completion and Applications

As an essential part of artificial intelligence, a knowledge graph describes the real-world entities, concepts and their various semantic relationships in a structured way and has been gradually popularized in a variety practical scenarios. The majority of existing knowledge graphs mainly concentrate on organizing and managing textual knowledge in a structured representation, while paying little attention to the multimodal resources (e.g., pictures and videos), which can serve as the foundation for the machine perception of a real-world data scenario. To this end, in this survey, we comprehensively review the related advances of multimodal knowledge graphs, covering multimodal knowledge graph construction, completion and typical applications. For construction, we outline the methods of named entity recognition, relation extraction and event extraction. For completion, we discuss the multimodal knowledge graph representation learning and entity linking. Finally, the mainstream applications of multimodal knowledge graphs in miscellaneous domains are summarized.

Automated summary

This survey comprehensively reviews the related advances of multimodal knowledge graphs, including their construction, completion, and typical applications. The methods of named entity recognition, relation extraction, and event extraction are outlined for construction, while multimodal knowledge graph representation learning and entity linking are discussed for completion. The mainstream applications of multimodal knowledge graphs in various domains are summarized.