tts-hifigan-libritts-22050Hz

This repository provides all the necessary tools for using a [HiFIGAN](https://arxiv.org/abs/2010.05646) vocoder trained with [LibriTTS](https://www.openslr.org/60/) (with multiple speakers). The sample rate used for the vocoder is 22050 Hz. The pre-trained model takes in input a spectrogram and produces a waveform in output. Typically, a vocoder is used after a TTS model that converts an input text into a spectrogram. Alternatives to this models are the following: - [tts-hifigan-libritts-16kHz](https://huggingface.co/speechbrain/tts-hifigan-libritts-16kHz/) (same model trained on the same dataset, but for a sample rate of 16000 Hz) - [tts-hifigan-ljspeech](https://huggingface.co/speechbrain/tts-hifigan-ljspeech) (same model trained on LJSpeech for a sample rate of 22050 Hz). pip install speechbrain Please notice that we encourage you to read our tutorials and learn more about [SpeechBrain](https://speechbrain.github.io). - *Basic Usage:* import torch from speechbrain.inference.vocoders import HIFIGAN hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-libritts-22050Hz", savedir="pretrained_models/tts-hifigan-libritts-22050Hz") mel_specs = torch.rand(2, 80,298) waveforms = hifi_gan.decode_batch(mel_specs) - *Spectrogram to Waveform Conversion:* import torchaudio from speechbrain.inference.vocoders import HIFIGAN from speechbrain.lobes.models.FastSpeech2 import mel_spectogram hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-libritts-22050Hz", savedir="pretrained_models/tts-hifigan-libritts-22050Hz") signal, rate = torchaudio.load('/home/mirco/Downloads/example_22kHz.wav') signal = signal[0].squeeze() torchaudio.save('waveform.wav', signal.unsqueeze(0), 22050) spectrogram, _ = mel_spectogram( audio=signal.squeeze(), sample_rate=22050, hop_length=256, win_length=1024, n_mels=80, n_fft=1024, f_min=0.0, f_max=8000.0, power=1, normalized=False, min_max_energy_norm=True, norm="slaney", mel_scale="slaney", compression=True ) waveforms = hifi_gan.decode_batch(spectrogram) torchaudio.save('waveform_reconstructed.wav', waveforms.squeeze(1), 22050) import torchaudio from speechbrain.inference.TTS import Tacotron2 from speechbrain.inference.vocoders import HIFIGAN tacotron2 = Tacotron2.from_hparams(source="speechbrain/tts-tacotron2-ljspeech", savedir="pretrained_models/tts-tacotron2-ljspeech") hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-libritts-22050Hz", savedir="pretrained_models/tts-hifigan-libritts-22050Hz") mel_output, mel_length, alignment = tacotron2.encode_text("Mary had a little lamb") waveforms = hifi_gan.decode_batch(mel_output) torchaudio.save('example_TTS.wav',waveforms.squeeze(1), 22050) To perform inference on the GPU, add `run_opts={"device":"cuda"}` when calling the `from_hparams` method. The model was trained with SpeechBrain. To train it from scratch follow these steps: 1. Clone SpeechBrain: git clone https://github.com/speechbrain/speechbrain/ 2. Install it: cd speechbrain pip install -r requirements.txt pip install -e . 3. Run Training: cd recipes/LibriTTS/vocoder/hifigan/ python train.py hparams/train.yaml --data_folder=/path/to/LibriTTS_data_destination --sample_rate=22050 To change the sample rate for model training go to the `"recipes/LibriTTS/vocoder/hifigan/hparams/train.yaml"` file and change the value for `sample_rate` as required. The training logs and checkpoints are available [here](https://drive.google.com/drive/folders/1cImFzEonNYhetS9tmH9R_d0EFXXN0zpn?usp=sharing).