DyGAT: Dynamic stroke classification of online handwritten documents and sketches

Online handwriting is widely used in human-machine interface, education, office automation, and so on. Stroke classification for online handwritten documents and sketches aims to divide strokes into several semantic categories and is a necessary step for document recognition and understanding. Previous methods are essentially static in that they have to wait for the user to finish the whole sketch before making prediction. However, in practice, the more user-friendly way is to make real-time prediction as the user is writing. In this paper, we introduce Dynamic Graph ATtention network (DyGAT) to solve the dynamic stroke classification problem. The core of our method is to formalize a document/sketch into a multifeature graph, in which nodes represent strokes, edges represent the relationships between strokes, and multiple nodes are applied to one stroke to control the information flow. The proposed method is general and is applicable to online handwritten data of many types. We conduct experiments on popular public datasets to perform sketch semantic segmentation, document layout analysis and diagram recognition, and experimental results show competitive performance. Particularly, the proposed method achieves stroke classification accuracies which are only slightly lower than those of static classification.(c) 2023 Elsevier Ltd. All rights reserved.

Automated summary

Online handwriting is widely used in various domains. This paper introduces a method called Dynamic Graph ATtention network (DyGAT) to solve the dynamic stroke classification problem. The core idea is to formalize a document/sketch as a multifeature graph with nodes representing strokes and edges representing their relationships. The proposed method is applicable to different types of online handwritten data and achieves competitive performance in various tasks.