4.8 Article

End-to-End Handwritten Paragraph Text Recognition Using a Vertical Attention Network

Publisher

IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2022.3144899

Keywords

Seq2Seq model; hybrid attention; segmentation-free; paragraph handwriting recognition; fully convolutional network; encoder-decoder; optical character recognition

Ask authors/readers for more resources

This paper proposes a unified end-to-end model using hybrid attention for unconstrained handwritten text recognition. By iteratively processing a paragraph image line by line, the model can generate vertical weighted masks to implicitly segment lines and recognize the character sequence of the whole paragraph.
Unconstrained handwritten text recognition remains challenging for computer vision systems. Paragraph text recognition is traditionally achieved by two models: the first one for line segmentation and the second one for text line recognition. We propose a unified end-to-end model using hybrid attention to tackle this task. This model is designed to iteratively process a paragraph image line by line. It can be split into three modules. An encoder generates feature maps from the whole paragraph image. Then, an attention module recurrently generates a vertical weighted mask enabling to focus on the current text line features. This way, it performs a kind of implicit line segmentation. For each text line features, a decoder module recognizes the character sequence associated, leading to the recognition of a whole paragraph. We achieve state-of-the-art character error rate at paragraph level on three popular datasets: 1.91% for RIMES, 4.45% for IAM and 3.59% for READ 2016. Our code and trained model weights are available at https://github.com/FactoDeepLearning/VerticalAttentionOCR.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.8
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available