kaen2891.github.io

Sample results from “Vocoder-free End-to-End Voice Conversion with Transformer Network”

Authors: June-Woo Kim, Ho-Young Jung, Minho Lee

Abstract: Mel-frequency filter bank (MFB) based approaches have the advantage of higher learning speeds compared to using the raw spectrum due to a smaller number of features. However, speech generators with the MFB approach require an additional computationally expensive vocoder for the training process. The pre- and post-processing needed by the MFB and the vocoder is not essential to convert human voices, because it is possible to use only the raw spectrum to generate different style of voices with clear pronunciation. In this paper, we introduce a vocoder-free end-to-end voice conversion method using a transformer network to alleviate the computational burden from additional pre- and post-processing. Our transformer-based architecture, which does not have any CNN or RNN layers, has shown the benefit of learning fast while solving the limitation of sequential computation of the conventional RNN. For this reason, our model is a fast and effective approach to convert realistic voices using raw spectra in a parallel manner to generate different style of voices with clear pronunciation. Furthermore, we can get an adapted MFB for speech recognition by multiplying the converted magnitude with the phase information, and therefore our conversion model is also suitable for speaker adaptation. We perform our voice conversion experiments on TIDIGITS-dataset using the naturalness, similarity, and clarity with Mean Opinion Score as metrics.

The first column is source voice, second is target voice, and last is converted voice.

First domain voice : Girl

Source: Girl, Target: Man, Result: Converted Man

Saying ‘Three’

Saying ‘Seven’

Saying ‘Nine’

Saying ‘Zero’

Source: Girl, Target: Woman, Result: Converted Woman

Saying ‘Two’

Saying ‘Five’

Saying ‘Eight’

Saying ‘Oh’

Source: Girl, Target: Boy, Result: Converted Boy

Saying ‘Four’

Saying ‘Five’

Saying ‘Six’

Saying ‘Eight’

Second domain voice : Boy

Source: Boy, Target: Man, Result: Converted Man

Saying ‘Four’

Saying ‘Six’

Saying ‘Nine’

Saying ‘Oh’

Source: Boy, Target: Woman, Result: Converted Woman

Saying ‘One’

Saying ‘Two’

Saying ‘Four’

Saying ‘Eight’

Source: Boy, Target: Girl, Result: Converted Girl

Saying ‘Two’

Saying ‘Five’

Saying ‘Eight’

Saying ‘Zero’

Third domain voice : Woman

Source: Woman, Target: Man, Result: Converted Man

Saying ‘One’

Saying ‘Two’

Saying ‘Three’

Saying ‘Nine’

Source: Woman, Target: Boy, Result: Converted Boy

Saying ‘Four’

Saying ‘Six’

Saying ‘Nine’

Saying ‘Zero’

Source: Woman, Target: Girl, Result: Converted Girl

Saying ‘Four’

Saying ‘Five’

Saying ‘Six’

Saying ‘Zero’

Fourth domain voice : Man

Source: Man, Target: Woman, Result: Converted Woman

Saying ‘Three’

Saying ‘Five’

Saying ‘Seven’

Saying ‘Oh’

Source: Man, Target: Boy, Result: Converted Boy

Saying ‘One’

Saying ‘Three’

Saying ‘Six’

Saying ‘Seven’

Source: Man, Target: Girl, Result: Converted Girl

Saying ‘Three’

Saying ‘Five’

Saying ‘Oh’

Saying ‘Zero’