Learning Hierarchical Document Graphs From Multilevel Sentence Relations

Organizing the implicit topology of a document as a graph, and further performing feature extraction via the graph convolutional network (GCN), has proven effective in document analysis. However, existing document graphs are often restricted to expressing single-level relations, which are predefined and independent of downstream learning. A set of learnable hierarchical graphs are built to explore multilevel sentence relations, assisted by a hierarchical probabilistic topic model. Based on these graphs, multiple parallel GCNs are used to extract multilevel semantic features, which are aggregated by an attention mechanism for different document-comprehension tasks. Equipped with variational inference, the graph construction and GCN are learned jointly, allowing the graphs to evolve dynamically to better match the downstream task. The effectiveness and efficiency of the proposed multilevel sentence relation graph convolutional network (MuserGCN) is demonstrated via experiments on document classification, abstractive summarization, and matching.

Automated summary

In this paper, a multilevel sentence relation graph convolutional network (MuserGCN) is proposed to analyze documents by organizing their implicit topology as a graph and performing feature extraction using graph convolutional networks. A set of learnable hierarchical graphs are constructed to explore multilevel sentence relations, and multiple parallel graph convolutional networks are used to extract multilevel semantic features, which are aggregated using an attention mechanism for different document comprehension tasks. Variational inference is used to learn the graph construction and evolve the graphs dynamically to better match downstream tasks.