期刊
SPEECH COMMUNICATION
卷 148, 期 -, 页码 53-65出版社
ELSEVIER
DOI: 10.1016/j.specom.2023.02.005
关键词
Representation learning; Deep generative models; Variational autoencoder; Source-filter model
Understanding and controlling latent representations in deep generative models is a challenging yet important problem. In this work, the source-filter model of speech production naturally arises as orthogonal subspaces of the VAE latent space. A method is proposed to identify and control the source-filter speech factors within the latent subspaces, as well as a robust f(0) estimation method.
Understanding and controlling latent representations in deep generative models is a challenging yet important problem for analyzing, transforming and generating various types of data. In speech processing, inspiring from the anatomical mechanisms of phonation, the source-filter model considers that speech signals are produced from a few independent and physically meaningful continuous latent factors, among which the fundamental frequency f(0) and the formants are of primary importance. In this work, we start from a variational autoencoder (VAE) trained in an unsupervised manner on a large dataset of unlabeled natural speech signals, and we show that the source-filter model of speech production naturally arises as orthogonal subspaces of the VAE latent space. Using only a few seconds of labeled speech signals generated with an artificial speech synthesizer, we propose a method to identify the latent subspaces encoding f(0) and the first three formant frequencies, we show that these subspaces are orthogonal, and based on this orthogonality, we develop a method to accurately and independently control the source-filter speech factors within the latent subspaces. Without requiring additional information such as text or human-labeled data, this results in a deep generative model of speech spectrograms that is conditioned on f(0) and the formant frequencies, and which is applied to the transformation speech signals. Finally, we also propose a robust f(0) estimation method that exploits the projection of a speech signal onto the learned latent subspace associated with f(0).
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据