The VoiceMOS Challenge 2022

We present the first edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthetic speech. This challenge drew 22 participating teams from academia and industry who tried a variety of approaches to tackle the problem of predicting human ratings of synthesized speech. The listening test data for the main track of the challenge consisted of samples from 187 different text-to-speech and voice conversion systems spanning over a decade of research, and the out-of-domain track consisted of data from more recent systems rated in a separate listening test. Results of the challenge show the effectiveness of fine-tuning self-supervised speech models for the MOS prediction task, as well as the difficulty of predicting MOS ratings for unseen speakers and listeners, and for unseen systems in the out-of-domain setting.

Automated summary

The VoiceMOS Challenge aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthetic speech. Through this challenge, 22 participating teams from academia and industry tested various approaches to predict human ratings of synthesized speech. The results highlight the effectiveness of fine-tuning self-supervised speech models for MOS prediction, as well as the challenges in predicting MOS ratings for unseen speakers, listeners, and systems in the out-of-domain setting.