A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection

A great deal of recent research effort on speech spoofing countermeasures has been invested into back-end neural networks and training criteria. We contribute to this effort with a comparative perspective in this study. Our comparison of countermeasure models on the ASVspoof 2019 logical access scenario takes into account common strategies to deal with input trials of varied length, recently proposed margin-based training criteria, and widely used front ends. We also measured intra-model differences through multiple training-evaluation rounds with random initialization. Our statistical analysis demonstrates that the performance of the same model may be statistically significantly different when just changing the random initial seed. We thus recommend similar statistical analysis or reporting results of multiple runs for further research on the database. Despite the intra-model differences, we observed a few promising techniques, including average pooling, to efficiently process varied-length inputs and a new hyper-parameter-free loss function. The two techniques led to the best single model in our experiment, which achieved an equal error rate of 1.92% and was significantly different in statistical sense from most of the other experimental models.

Automated summary

Recent research has focused on back-end neural networks and training criteria for speech spoofing countermeasures. This study offers a comparative perspective on various models and recognizes the potential impact of random initial seed on model performance. Promising techniques, including average pooling and a new hyper-parameter-free loss function, led to the best single model with significantly different statistical performance compared to others.