ITALIAN-LEGAL-BERT models for improving natural language processing tasks in the Italian legal domain

Legal-BERT models are based on the BERT architecture (or its variants) and have been developed specifically for the legal domain. They have reached the state of the art in complex legal tasks such as legal research, document synthesis, contract analysis, argument extraction, and legal prediction. In this paper, we proposed four versions of Legal-BERT models pre-trained on the Italian legal domain. They aim to improve NLP applications in the Italian legal context. We have shown that they outperforms the Italian generalpurpose BERT in several domain-specific tasks, such as named entity recognition, sentence classification, semantic similarity with Bi-encoders, and document classification.

Automated summary

Legal-BERT models, based on the BERT architecture, have been developed specifically for the legal domain and achieved state-of-the-art performance in complex legal tasks. This paper proposes four versions of Legal-BERT models pre-trained on the Italian legal domain, showing their superiority in several domain-specific tasks.