Speech Time-Scale Modification with GANs

Eyal Cohen, Felix Kreuk, Joseph Keshet

Research output: Contribution to journalArticlepeer-review


While listening to spoken content, it is often desired to vary the speech rate while preserving the speaker's timbre and pitch. To date, advanced signal processing techniques are used to address this task, but it still remains a challenge to maintain a high speech quality at all time-scales. Inspired by the success of speech generation using Generative Adversarial Networks (GANs), we propose a novel unsupervised learning algorithm for time-scale modification (TSM) of speech, called ScalerGAN. The model is trained using a set of speech utterances, where no time-scales are provided. The ScalerGAN algorithm is composed of a generator that gets as input speech with the desired rate and outputs a time-adjusted speech; a discriminator that works on various spectrum scales; and a decoder that converts the time-adjusted signal back to the original rate to maintain consistency. Using an A/B test and conditional A/B test, human listeners were asked to compare ScalerGAN with other state-of-the-art TSM methods. The results showed that the speech quality of ScalerGAN outperforms all other methods.

Original languageEnglish
Pages (from-to)1067-1071
Number of pages5
JournalIEEE Signal Processing Letters
StatePublished - 2022

Bibliographical note

Publisher Copyright:
© 2021 IEEE.


  • Deep neural networks
  • generative adversarial networks
  • speech synthesis
  • time-scale modification


Dive into the research topics of 'Speech Time-Scale Modification with GANs'. Together they form a unique fingerprint.

Cite this