speecht5-tts-turkish

Turkish text-to-speech, fine-tuned from [`microsoft/speecht5_tts`](https://huggingface.co/microsoft/speecht5_tts) on [`erenfazlioglu/turkishvoicedataset`](https://huggingface.co/datasets/erenfazlioglu/turkishvoicedataset). Paired with the [`microsoft/speecht5_hifigan`](https://huggingface.co/microsoft/speecht5_hifigan) vocoder it converts Turkish text into a 16 kHz waveform. The base checkpoint ships a 79-piece character SentencePiece model built for English. It contains none of `ç ğ ı İ ö ş ü` and **no digits**, so 99.55% of the Turkish corpus contained at least one character it could not represent. This model therefore uses an **extended tokenizer** (99 tokens): the Turkish letters were appended as real SentencePiece pieces, and the text embedding was grown from 81 to 99 rows with every pre-trained vector carried over and each new letter seeded from a related base letter. **The processor bundled with this repository is required.** Loading these weights with the base `microsoft/speecht5_tts` processor will produce wrong output. Numbers must still be spelled out — the vocabulary has no digits. The inference code does this automatically; if you call the model directly, expand numbers to Turkish words first (`2026` → `iki bin yirmi altı`). import numpy as np import soundfile as sf import torch from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor repo = "thealper2/speecht5-tts-turkish" processor = SpeechT5Processor.from_pretrained(repo) model = SpeechT5ForTextToSpeech.from_pretrained(repo) vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan") from huggingface_hub import hf_hub_download speaker = np.load(hf_hub_download(repo, "speaker_embedding.npy")) speaker = torch.from_numpy(speaker).unsqueeze(0) text = "Merhaba, bu Türkçe SpeechT5 modelinin test cümlesidir." inputs = processor(text=text, return_tensors="pt") speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder) sf.write("output.wav", speech.numpy(), 16000) For number handling, punctuation normalisation and long-text chunking, use the repository's `src/inference.py` helper instead of calling the model directly. The corpus is a single synthetic voice. Measured across 1,500 random clips, RMS standard deviation is 0.008 and x-vector cosine similarity to the corpus centroid averages 0.991 (minimum 0.940). Split (deterministic, seed 42, duplicate-group aware so no near-duplicate family straddles a boundary): **117,625 train / 3,091 validation / 3,108 test**, verified at 0 cross-split transcription collisions. Preprocessing: silence trimmed at 45 dB, audio capped at 12 s, text capped at 450 tokens (rows exceeding the cap are dropped rather than truncated), numbers expanded to Turkish words, typographic punctuation mapped onto the tokenizer's ASCII inventory. Objective scoring uses `text → this model → openai/whisper-small → text`, over a fixed 30-sentence Turkish set covering short, medium, long, Turkish-character-heavy, numeric and punctuation-heavy inputs. Per-category breakdown of the fixed evaluation set: Audio health across the same set: mean 4.07 s, longest internal gap 2.23 s, 0 silent outputs. ASR error rates are **auxiliary**. The recogniser has its own error floor on this corpus (reported above); scores should be read against that floor, not against zero. An ASR mistake is not proof of a TTS mistake. The fine-tuned WER landing close to the ASR floor does **not** mean the synthesis is as good as the original recordings. The two figures are measured on different text: the floor comes from real corpus sentences (news copy, dense with proper nouns), while the evaluation set is 30 hand-written sentences chosen to probe specific behaviours. Treat the floor as a scale marker, not as a target that has been beaten. Some remaining errors are scoring artefacts rather than pronunciation faults — the recogniser writes `cm` for a spoken *santimetre*, and drops the circumflex in `rüzgâr` and `zekâ`. The true error rate is therefore somewhat lower than the table shows. **No MOS score is reported. No MOS study was conducted.** Any subjective ratings in the project report are informal listening notes by a single listener, labelled as such. * **Turkish-specific letters are the weakest point.** The `turkish_chars` category scores roughly four times worse than every other category. Errors concentrate on `ğ` (`Iğdır` → *idlir*, `öğleden` → *o neden*) and on word-initial `ş` (`şoförüyle` → *sofori*). This is expected: those characters were added to the vocabulary for this fine-tune and had two epochs to train, against the base alphabet's full pre-training. `ğ` is additionally the hardest case in Turkish for any TTS, since it marks vowel lengthening rather than a segment of its own. More epochs, or oversampling `ğ`-dense sentences, is the obvious next experiment. * **Synthetic voice ceiling.** The training audio was produced by a TTS engine, so this model reproduces that engine's timbre, speaking rate and prosody. It will not sound like a recorded human speaker, and the gap is a property of the data, not of the training run. * **Single speaker.** No voice cloning or multi-speaker control. Substituting a different x-vector is out of distribution and degrades quality. * **Numbers and abbreviations must be expanded** before tokenisation; the vocabulary has no digits. * **Domain skew.** The corpus is predominantly Turkish news text, including politically charged material. Prosody is tuned to news-reading register. * **Long input.** Quality degrades on very long single utterances; chunk on sentence boundaries. * **Non-Turkish text** is out of scope — the tokenizer covers the Turkish alphabet plus basic Latin. The training dataset is licensed **CC BY-NC 4.0 (non-commercial)**, and fine-tuning does not remove that restriction. This repository therefore does **not** claim the weights are suitable for commercial use. Before redistributing or using this model commercially, review: 1. the `erenfazlioglu/turkishvoicedataset` licence (CC BY-NC 4.0), 2. Microsoft's terms for Text-to-Speech output, since the training audio was generated by that service, 3. the MIT licence of the `microsoft/speecht5_tts` base model. Intended for research and educational use. @inproceedings{ao2022speecht5, title = {SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing}, author = {Ao, Junyi and Wang, Rui and Zhou, Long and Wang, Chengyi and Ren, Shuo and Wu, Yu and Liu, Shujie and Ko, Tom and Li, Qing and Zhang, Yu and Wei, Zhihua and Qian, Yao and Li, Jinyu and Wei, Furu}, booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics}, year = {2022} }