SVSNet: An End-to-End Speaker Voice Similarity Assessment Model

Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention. In this paper, we propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between converted speech and natural speech for voice conversion tasks. Unlike most neural evaluation metrics that use hand-crafted features, SVSNet directly takes the raw waveform as input to more completely utilize speech information for prediction. SVSNet consists of encoder, co-attention, distance calculation, and prediction modules and is trained in an end-to-end manner. The experimental results on the Voice Conversion Challenge 2018 and 2020 (VCC2018 and VCC2020) datasets show that SVSNet outperforms well-known baseline systems in the assessment of speaker similarity at the utterance and system levels.

Automated summary

This paper introduces SVSNet, the first end-to-end neural network model for assessing speaker voice similarity in voice conversion tasks. Unlike most neural evaluation metrics, SVSNet takes raw waveform as input to make full use of speech information. Experimental results on VCC2018 and VCC2020 datasets show that SVSNet outperforms baseline systems in assessing speaker similarity at both utterance and system levels.