A multi-task approach for contrastive learning of handwritten signature feature representations

In spite of recent advances in computer vision, the classic problem of offline handwritten signature verification still remains challenging. The signature verification task has a high intra-class variability because a given user often shows high variability between its samples. Besides, signature verification is harder in the presence of skilled forgeries. Recently, in order to tackle these challenges, the research community has investigated deep learning methods for learning feature representations of handwritten signatures. When mapping signatures to a feature space, it is desired to obtain dense clusters of signature's representations, in order to deal with intra-class variability. Besides, not only dense clusters are required but also a larger separation between different user's clusters in the feature space. Finally, it is also desired to move away feature representations of skilled forgeries in relation to the respective dense cluster of genuine representations. This last property is hard to achieve in the real-world scenario because skilled forgeries are not readily available during training. In this work, we hypothesize that such properties can be achieved by means of a multi-task framework for learning handwritten signature feature representations based on deep contrastive learning. The proposed framework is composed of two objective-specific tasks. The first task aims to map signature examples of the same user closer within the feature space, while separating the feature representations of signatures of different users. The second task aims to adjust the skilled forgeries representations by adopting contrastive losses with the ability to perform hard negative mining. Hard negatives are examples from different classes with some degree of similarity that can be applied for training. We evaluated models obtained with the proposed framework in terms of the equal error rate on GPDSsynthetic, CEDAR and MCYT-75 datasets in writer-dependent and writer-independent verification approaches. Using synthetic and real signature datasets, Friedman tests with Bonferroni-Dunn post hoc tests were performed to compare the proposed multi-task contrastive models against the popular SigNet model as a baseline. Experiments demonstrated an statistically significant improvement in signature verification with a multi-task contrastive model based on the Triplet loss. Implementation of the method is available for download at https://github.com/tallesbrito/contrastive_sigver.

Automated summary

Despite recent advances in computer vision, the problem of offline handwritten signature verification remains challenging. Deep learning methods have been investigated to learn feature representations of handwritten signatures. A multi-task framework based on deep contrastive learning is proposed to improve signature verification by adjusting the feature representations of genuine and skilled forgery signatures.