cosyvoice3-0.5b-2512-GGUF

GGUF conversion of [FunAudioLLM/Fun-CosyVoice3-0.5B-2512](https://huggingface.co/FunAudioLLM/Fun-CosyVoice3-0.5B-2512) for use with [CrispASR](https://github.com/CrispStrobe/CrispASR) (`--backend cosyvoice3-tts`). CosyVoice3 is a streaming, multilingual, zero-shot voice-cloning TTS system from Alibaba's FunAudioLLM team. The 0.5B-2512 release is Apache-2.0 licensed and supports **9 languages plus 18 Chinese dialects**. Output is 24 kHz mono. The model is a three-stage pipeline: text (Qwen2 BPE) → CosyVoice3LM (Qwen2-0.5B + speech-token AR head) → speech tokens ∈ [0, 6561) → Flow (DiT + CausalConditionalCFM, 10-step Euler ODE) → mel @ 24 kHz / 480-hop → CausalHiFTGenerator (HiFi-GAN + NSF + iSTFT) → 24 kHz PCM Upstream ships two talker checkpoints in one repo: `llm.pt` (pre-trained) and `llm.rl.pt` (the same architecture after reinforcement learning, tuned by the authors for speech quality, pronunciation accuracy and generation stability). Only the talker differs — **flow, HiFT, CAMPPlus, the speech tokenizer and the voice bank are shared**, so switching is just pointing `-m` at the other LLM GGUF and leaving every companion file in place: crispasr -m cosyvoice3-llm-rl-q4_k.gguf --backend cosyvoice3-tts \ --voice fleurs-en --i-have-rights \ --tts "The northern lights can be heard as well as seen." --tts-output out.wav CrispASR also ships `--backend cosyvoice3-tts-rl` (aliases `cosyvoice3-rl`, `cv3-rl`): the same engine, wired so that `-m auto` fetches the RL talker and reuses the shared companions. crispasr --backend cosyvoice3-tts-rl -m auto \ --voice fleurs-en --i-have-rights \ --tts "The northern lights can be heard as well as seen." --tts-output out.wav Both talkers are published here; keep whichever you prefer and delete the other. Pick **one LLM + one flow + HiFT + voices**. The smallest viable combo is `llm-q4_k + flow-q8_0 + hift-f16 + voices` at **745 MB total**; the F16 reference is 1.96 GB. The s3tok + campplus companions are only needed for **arbitrary-WAV runtime cloning** (below) — not for synthesis with a baked voice. Synthesis used the default zero-shot voice (upstream `asset/zero_shot_prompt.wav`) at `--temperature 0.8 --seed 42`. The generated WAV was transcribed with `parakeet-tdt-0.6b-v3-q4_k` and compared against the prompt text. Q4_K LLM introduces a small punctuation drift (commas occasionally read as question-intonation) but content is fully preserved across languages. Q8_0 flow is perceptually indistinguishable from F16. crispasr -m cosyvoice3-llm-q4_k.gguf \ --backend cosyvoice3-tts \ --tts "Hello, this is a test." \ --voice zero_shot \ --tts-output out.wav The CLI auto-discovers companion GGUFs in this order: * **Flow** — `cosyvoice3-flow-*.gguf` next to the LLM, or `--codec-model PATH`. * **CAMPPlus** — `cosyvoice3-campplus-f16.gguf` next to the LLM, or `COSYVOICE3_CAMPPLUS_PATH` for the native WAV-clone path. * **S3Tokenizer** — `cosyvoice3-s3tok-f16.gguf` next to the LLM, or `COSYVOICE3_S3TOK_PATH` for the native WAV-clone path. * **HiFT** — `cosyvoice3-hift-*.gguf` next to the LLM, or `COSYVOICE3_HIFT_PATH` env var. * **Voices** — `cosyvoice3-voices.gguf` next to the LLM, or `COSYVOICE3_VOICES_PATH` env var. Greedy decode is disabled by default (CV3 falls into a documented "silent_tokens" loop within ~5 steps). The backend overrides `--temperature 0` to 0.8 so the RAS sampler engages — pass a different positive value to override. `cosyvoice3-voices.gguf` ships a small multilingual voice bank — pass the name to `--voice`: The `fleurs-*` prompts are ~4–6 s clips from Google's [FLEURS](https://huggingface.co/datasets/google/fleurs) corpus (CC BY 4.0), loudness-normalised before baking. CV3 clones the prompt's timbre *and* level, so quiet prompts yield quiet output — normalise your own prompt clips for a consistent level. More voices can be baked with the converter in the CrispASR tree: python models/convert-cosyvoice3-voices-to-gguf.py \ --manifest my-voices.json \ --upstream-base /path/to/CosyVoice-clone \ --output my-voices.gguf \ --i-have-rights Each manifest entry is `{name, wav, prompt_text}`. The script needs `campplus.onnx` (CV2/CV3 speaker encoder) and `speech_tokenizer_v3.onnx` (CV3 token extractor); both auto-download from HF on first run. `--i-have-rights` is required: baking turns a real speaker's recording into a reusable voice clone, and by passing it you attest that you have that speaker's consent or that the voice is your own. The converter stamps each entry as recording-derived, which is what makes CrispASR demand `--i-have-rights` / `consent_attestation` at synthesis time and prepend the spoken AI disclosure to the output (EU AI Act Art. 50(4)). A bundle baked before that stamp existed still gates — every entry falls back to the producer architecture — but re-bake it for per-entry accuracy. With the `cosyvoice3-s3tok-f16.gguf` + `cosyvoice3-campplus-f16.gguf` companions present (siblings of the LLM, or pulled by `-m auto`), you can clone from any 16 kHz WAV at runtime: crispasr -m cosyvoice3-llm-q4_k.gguf \ --backend cosyvoice3-tts \ --voice my_reference.wav \ --ref-text "exact transcription of my_reference.wav" \ --tts "The text to speak in the cloned voice." \ --tts-output out.wav The runtime ports all three front-end extractors to ggml: the **speech_tokenizer_v3** token extractor (12 FSMN/attention blocks + FSQ head — byte-exact vs the ONNX reference, validated stage-by-stage with `crispasr-diff`), the **CAMPPlus** 192-D speaker encoder, and the **matcha 24 kHz** reference mel. The legacy Python pre-bake bridge (`convert-cosyvoice3-voices-to-gguf.py`) remains as an automatic fallback when the companions are absent. Conventional naming for all three GGUFs: * **LLM** — llama.cpp-standard `token_embd`, `blk.K.{attn,ffn}_*`, `output_norm`, `output`, plus CV3-specific `cosyvoice3.speech_embd.weight` (input embedding, vocab 6761) and `cosyvoice3.speech_lm_head.weight` (output head). * **Flow** — `cosyvoice3.flow.{input_embd,pre_la,spk_affine,dit.*}` matching the upstream `CausalMaskedDiffWithDiT` module tree. * **HiFT** — `cosyvoice3.hift.{conv_pre,ups.K,resblocks.K.*,source_*, m_source,f0.*,conv_post}` with weight-norm pre-resolved on the Python converter side (g · v / ‖v‖). The model weights are **Apache-2.0** (inherited from the upstream model). Free for commercial use. The `zero_shot` voice prompt is the `asset/zero_shot_prompt.wav` clip from the Apache-2.0 CosyVoice repo. The `fleurs-{en,de,zh,ja,fr,es,ko}` voice prompts are derived (trimmed + loudness-normalised) from Google's **FLEURS** corpus, licensed **[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)** — commercial use permitted, attribution required: > FLEURS (Few-shot Learning Evaluation of Universal Representations of > Speech), Conneau et al., 2022 — <https://huggingface.co/datasets/google/fleurs>, > licensed CC BY 4.0. The prompt clips here are trimmed excerpts, > loudness-normalised; no other modification. All eight baked voices are therefore clean for commercial use under permissive licenses (Apache-2.0 / CC BY 4.0). * Upstream: [FunAudioLLM/Fun-CosyVoice3-0.5B-2512](https://huggingface.co/FunAudioLLM/Fun-CosyVoice3-0.5B-2512) * Project page: [funaudiollm.github.io/cosyvoice3](https://funaudiollm.github.io/cosyvoice3/) * Code: [github.com/FunAudioLLM/CosyVoice](https://github.com/FunAudioLLM/CosyVoice) * CrispASR: [github.com/CrispStrobe/CrispASR](https://github.com/CrispStrobe/CrispASR)