Chatterbox is Resemble AI’s open text-to-speech model, released under MIT. It does zero-shot voice cloning: give it a few seconds of reference audio and it speaks new text in that voice, with no training step. That is a step beyond the fixed voices in basic text-to-speech. The one rule that comes first: only clone voices you have permission to use. The full usage is on the Chatterbox model card.
1. How zero-shot voice cloning works
A zero-shot model conditions on a short reference clip at generation time instead of being fine-tuned per speaker. You hand it the clip as audio_prompt_path, and it copies that voice’s timbre and delivery onto whatever text you pass. Quality tracks the reference: clean, dry mono audio beats a noisy phone recording. To go the other direction and turn recordings into text, see transcribe audio with Whisper.
2. Install Chatterbox
pip install chatterbox-tts
Pulls in the model code plus torch and torchaudio; a CUDA GPU makes generation far faster than CPU.
3. Prepare a reference clip
Record or trim a short clip of the target voice speaking clearly, saved as a WAV (the official examples use a roughly ten-second file). Keep it mono and free of music or background noise, because the model imitates whatever it hears, artefacts included.
4. Generate speech in that voice
import torchaudio as ta
from chatterbox.tts import ChatterboxTTS
model = ChatterboxTTS.from_pretrained(device="cuda")
text = "Hello, this sentence is spoken in the cloned voice."
wav = model.generate(text, audio_prompt_path="reference.wav")
ta.save("output.wav", wav, model.sr)
Loads the model, conditions on reference.wav, and writes output.wav at the model’s own sample rate, read from model.sr.
5. Consent, length, and the watermark
Generate a sentence to a few sentences per call; split longer text and concatenate the clips. Every Chatterbox output carries Resemble’s built-in Perth (Perceptual Threshold) Watermarker, so synthetic audio stays detectable. The ethics are not optional: clone only your own voice or one you have explicit, documented permission to use, and never impersonate a real person.
Verify it worked
Play output.wav: it should read your text in the reference voice. If it sounds like a generic default voice, the reference clip was not applied.
If it fails:
ModuleNotFoundError: No module named 'chatterbox'— the package ischatterbox-tts; reinstall it under that name.torch.cuda.OutOfMemoryErroror no GPU — passdevice="cpu"tofrom_pretrained(slower, but it runs).- Robotic or wrong-sounding voice — use a cleaner, longer mono reference; noise and music degrade the clone.