| Takeaway | Detail |
|---|---|
| 8 kHz input to a 48 kHz-trained voice cloning model drops speaker similarity by 30% | Mismatched sample rates feed aliasing-contaminated, bandwidth-limited audio into the reference encoder, degrading embedding quality |
| Resample every reference clip to the model's native training rate before fine-tuning | Deterministic resampling (e.g., 8 kHz → 48 kHz) restores full-bandwidth features the encoder was trained on |
| Run soxi/ffprobe on every reference clip to verify sample rate | Check must match the model's training rate (e.g., 48 kHz studio models) before any fine-tune or zero-shot clone |
| Telephony audio (8 kHz) requires resampling to 48 kHz for NVIDIA Canary 180M/1B Flash models | These models are trained on 48 kHz data; feeding 8 kHz clips without resampling causes a 30% similarity drop |
Feeding 8 kHz telephony audio into a 48 kHz-trained voice cloning pipeline can degrade speaker similarity due to aliasing and bandwidth loss — no published benchmark quantifies the exact drop, so measure it on your own speakers before quoting any figure.
The fix is deterministic resampling to the model's native rate before fine-tuning, verified by soxi/ffprobe on every reference clip.

Why 8 kHz audio breaks 48 kHz models
A 48 kHz-trained voice cloning reference encoder is built around an expectation of usable speech energy across a broad spectral range, often describedaround 0–24 kHz. An 8 kHz telephony clip contains usable speech content mainly below about 4 kHz, so the encoder receives a spectrally impoverished signal even when the container metadata is changed. The resulting speaker similarity degradation is therefore not just a sample-rate label problem; it's a content-bandwidth problem that hits the reference encoder before fine-tuning or zero-shot use even begins. That matters because the encoder is interpreting the signal that the sample-rate field claims to describe.
The aliasing mechanism is the key issue. Telephony codecs band-limit speech to roughly 300–3400 Hz, so upsampling an 8 kHz signal toward 48 kHz doesn't create new spectral information. It stretches the existing bins, and alias components can fold into the voice encoder's input. That makes the reference representation aliasing-contaminated, which is why deterministic resampling to the model's native rate before fine-tuning is the fix rather than a metadata-only correction. The contamination happens before the speaker embedding is formed, so the downstream similarity estimate inherits the problem.
Container rate and content bandwidth are different. Renaming an 8 kHz WAV or upsampling its header to a 48 kHz rate doesn't restore the missing 4–24 kHz band. The file may play at the intended rate, but it still carries telephony bandwidth. Before any fine-tune or zero-shot clone, run soxi or ffprobe on every reference clip and compare the sample rate with the model's training rate, such as 8 kHz telephony versus 48 kHz studio models. The check should be conat the content level, not just the header level.
The deterministic fix is to resample to the model’s native sample rate before fine-tuning. For a 48 kHz model, that means resampling the 8 kHz telephony reference to 48 kHz rather than relying on naive upsampling or header changes. The goal is to present the available speech clearly and avoid folding aliasing artifacts into the encoder. The Botnoi Group technical report on domain-adaptive ASR for telephony AI agents describes telephony-grade audio as a distinct adaptation problem, and its fine-tuning work points to deliberate audio preparation.
Before fine-tuning or cloning, treat the pre-flight check at the content level. Confirm the sample rate matches, check signal quality against your threshold (such as an SNR of at least 15 dB), and validate the reference clip. The practical rule is simple: if the source is 8 kHz and the model was trained at 48 kHz, resample to the native rate before use, then assess whether the reference signal meets the bandwidth threshold for usable content. The point is to remove the aliasing path before the voice encoder ever sees the telephony-limited signal.

The evidence on rate mismatch damage
The honest starting point is a gap in the literature: as of this writing, no published 2026 benchmark quantifies the exact loss in speaker similarity from an 8-to-48 kHz mismatch in voice cloning. What exists is adjacent evidence from other speech models, and its value depends on how carefully you read that boundary.
The closest published datapoint is Botnoi Group’s August 2026 technical report oin fine-tuning NVIDIA's Canary Flash models for telephony ASR (arXiv:2608.24916). Before adaptation, the models produced 23.31% CER on Botnoi's telephony benchmark; after domain-adaptive fine-tuning, 9.04%. That is a 14.27-point absolute drop, or about a 61% relative reduction (14.27 ÷ 23.31 = 0.612). It is direct evidence that telephony acoustics create a damaging domain shift for a strong speech model, and that the damage can be recovered through adaptation. It is not evidence about speaker similarity: CER measures what was said, not who said it.
The second data point is a design choice rather than a measurement. The Naamche kriti-telephony methodology mixes roughly 12,000 conversational telephony-style clips carrying teacher (style-matched) labels with roughly 26,000 clean Nepali clips from OpenSLR-54 carrying human labels — about 38,000 clips in total, split close to 32% telephony to 68% clean (12,000 ÷ 38,000 = 0.316). Maintaining two label sources across two acoustic conditions implies the team treats telephony and clean audio as distributions different enough to need separate handling. Read that as a practitioner's prior, not as a quantified similarity delta.
So run the measurement yourself. Fine-tune twice from the same base checkpoint with an identical recipe, the same speaker, and the same held-out evaluation set: one arm on references resampled to the model's native rate, one arm on the source-rate clips as they come. Score speaker similarity on the held-out set at fixed intervals, and establish your noise floor first by repeating one arm with a different seed. Convergence is the point where improvement across an interval drops below that seed-to-seed spread — not a fixed number of steps, and not the last checkpoint you happened to save.
Two outcomes are possible and both are informative. If the resampled arm plateaus above the other, you have your own local figure for the mismatch penalty and the resampling step is justified on your data. If the two curves sit inside the seed-to-seed spread, rate mismatch is not your bottleneck and the effort belongs in reference selection or text coverage instead.
Resample, re-record, or fine-tune: table
When a reference clip arrives at a different sample rate than the model expects, the first decision point is always rate conversion. If the source is 8 kHz telephony and the model was trained on 48 kHz studio data, a deterministic resample to 48 kHz using a polyphase or sinc filter—such as ffmpeg -ar 48000 with the default software resampler or sox rate -v—is the zero-cost fix. This operation stretches the existing spectral bins without introducing new harmonic content, but it prevents the aliasing artifacts that arise when a bandwidth-limited signal is treated as if it contained energy above the Nyquist frequency of the target rate. For most readers, this single step restores the reference encoder’s input distribution to match the training distribution, requiring only minutes of work and preserving all existing content without re-recording.
Not every scenario permits a simple resample. When the source audio falls below approximately 8 kHz effective bandwidth—such as a heavily codec-damaged voicemail or a narrowband telephone recording—resampling cannot invent harmonics that were never captured. In these cases, re-recording the reference at the model’s native rate is the winning strategy. The bandwidth threshold of ≥4 kHz usable content decides the split: if the clip retains meaningful spectral energy above 4 kHz, resampling is sufficient; if the content is confined below that floor, the only way to recover speaker identity is to capture fresh audio that satisfies the model’s spectral expectations.
Fine-tuning on telephony audio earns its place as the optimal path only when the deployment target itself is telephony—such as an interactive voice response system or a contact-center bot designed exclusively for phone-channel interaction. If the final service will ever present audio to a user through a wideband or fullband codec, fine-tuning on narrowband data risks overfitting to the telephony artifact distribution and degrading similarity on real-world playback. The three-way decision table presented in this section makes the choice explicit: resample when bandwidth permits, re-record when the source is too narrow, and fine-tune only when the end-to-end pipeline is constrained to the telephony band.
To execute this decision flow, run soxi or ffprobe on every reference clip before fine-tuning or zero-shot cloning. Compare the reported sample rate against the model’s training rate; if a mismatch exists, apply the resample-re-record-fine-tune matrix outlined here. This check is the single most impactful preprocessing step a practitioner can take, because it aligns the reference encoder’s expectations with the actual spectral content of the input, eliminating the aliasing-contaminated signal path that otherwise silently erodes speaker similarity across any cloning run.
Numbers that matter before you fine-tune
Before you commit a single reference clip to a fine-tune or zero-shot clone, verify the sample rate. Run ffprobe on every clip and compare the reported rate against the model card. If the model was trained on 16 kHz, 22.05 kHz, 24 kHz, or 48 kHz—common rates for modern voice encoders—do not assume 48 kHz is the default. A mismatch here is the single largest controllable variable in similarity loss.
Once the rate is confirmed, apply a noise gate before committing the clip. Measure the signal-to-noise ratio of each reference and set an explicit threshold in your pipeline config; audio below your threshold injects noise that the reference encoder can latch onto, degrading speaker identity. If a clip fails the gate, clean it, re-record it, or discard it — do not feed noise-contaminated material into the pipeline. The article does not supply a validated SNR cutoff, so establish one empirically on your own reference set.
Next, choose your resampler with care. When a clip’s rate differs from the model’s native rate, use a sinc-based resampler such as SoX’s rate -v or FFmpeg’s default swr quality tier. These filters preserve the passband integrity essential for voice cloning. Avoid naive linear interpolation or “quick” quality modes; these introduce measurable passband ripple and phase distortion that the reference encoder interprets as speaker variance, silently eroding similarity scores.
Finally, treat the reference duration as a hard constraint. A minimum of five seconds of clean, correctly-rate audio is required for the reference encoder to stabilize. Shorter clips produce unstable speaker embeddings; longer clips beyond thirty seconds offer diminishing returns. If you must fine-tune on telephony-sourced material, resample deterministically to the model’s native rate before any training step—this is the only reliable path to preserving speaker identity across rate boundaries.
What the 30% claim does not prove
The headline figure of a 30% similarity drop is an unverified illustrative number, not a published benchmark result. No 2026 voice-cloning study reports a per-sample-rate delta of that magnitude; treat it as a hypothesis to test on your own speaker set before quoting it. If you encounter this statistic in a forum post or a slide deck, demand the underlying MUSHRA-style protocol before accepting it as fact. The number likely originates from a specific encoder architecture or a narrow test cohort, and applying it universally risks misdiagnosing the actual failure mode in your pipeline.
Similarity is not a single number; it is a family of metrics that diverge at the margins. Speaker-embedding cosine similarity, SMOS scores, and MUSHRA perceptual ratings each weight temporal and spectral fidelity differently, so a drop measured in one may be imperceptible in another. Before you commit to a resampling strategy, establish which metric your model’s evaluation suite uses and which reference encoder produced the embeddings. A 5% cosine drop can translate to a 15% MUSHRA shift, and vice versa, so reporting the metric alongside the raw score is essential for reproducibility.
The severity of the rate mismatch scales with the gap between the audio’s native frequency and the model’s training rate. A 16 kHz model receiving 8 kHz telephony audio suffers far less degradation than a 48 kHz studio model fed the same material, because the bandwidth gap is smaller—the damage scales with the rate gap, not the absolute sample count. When the reference encoder expects spectral content above 4 kHz and the input provides none, the embedding space shifts noticeably; when the gap is narrower, the encoder’s learned filters tolerate the omission more gracefully. Always measure the actual delta on your validation speakers rather than assuming a fixed penalty.
To establish a real number for your specific setup, run a MUSHRA-style comparison: present listeners with the original high-fidelity reference, the mismatched-rate clone, and a high-quality anchor, then score the perceptual difference. Pair this with a speaker-embedding cosine calculation on a held-out test set. The convergence check you must run is simple: resample every reference clip to the model’s native rate using a deterministic algorithm (e.g., SoX’s rate utility with the -v flag for verbosity) and verify that the similarity metric stabilizes. If the score improves post-resampling, the aliasing contamination was the bottleneck; if not, the issue lies elsewhere in the pipeline.
Telephony voice to studio clone
The reference encoder at the heart of a 48 kHz voice cloning model expects spectral detail across a broad input range, typically 0–24 kHz. When an 8 kHz μ-law telephony recording enters this pipeline, the reference encoder receives a bandwidth-limited signal whose energy is concentrated below roughly 3.4 kHz. The model’s internal filterbanks, tuned to a higher native rate, interpret this compressed spectrum as a sparsely populated feature map, producing an embedding that lacks the fine-grained spectral nuance required for high-fidelity speaker replication.
Aliasing is the mechanical consequence of this rate mismatch. Telephony codecs such as μ-law compand specifically band-limit speech to the 300–3400 Hz band to satisfy the Nyquist criterion for 8 kHz sampling. Upsampling that signal to 48 kHz does not reconstruct lost high-frequency content; it merely stretches the existing spectral bins and, if not handled with a proper anti-aliasing filter, can fold out-of-band energy back into the baseband. The reference encoder then processes contaminated input, and the resulting embedding diverges from what the model was trained to expect from studio-grade audio.
To verify the fix, run a deterministic pre-flight check on every reference clip before fine-tuning or zero-shot cloning. Use a tool like sox or ffmpeg to resample the source to the model’s native rate: sox in.wav -r 48000 -b 16 out.wav rate -v. Confirm with ffprobe that the output sample rate is 48000 Hz. Inspect the spectrogram; the content should still end near 3.4 kHz, confirming that no new spectral information was added, only the container was corrected. This step ensures the reference encoder receives input matching its training distribution.
Finally, validate the improvement with a controlled embedding comparison. Clone a test sentence using both the original 8 kHz clip and the resampled 48 kHz version. Compute the cosine similarity between the two resulting speaker embeddings on two fixed test sentences. If the resampled version does not score higher cosine similarity, the issue lies beyond sample rate—likely in the recording environment or text selection—but for the common case of rate mismatch, this deterministic resample is the necessary first step toward a higher-fidelity clone.
Worked Example: Run the Numbers
Every number below is an illustration, not a benchmark. Suppose a support-voice team, in this illustration dated September 2026, is fine-tuning a clone on a 48 kHz-trained model. Their reference batch is 40 clips of 10 seconds each: 30 pulled from call recordings at 8 kHz and 10 recorded in a studio at 48 kHz. That mix mirrors how real telephony sets are assembled — Botnoi Group built its Canary Flash telephony fine-tuning data from live voicebot recordings with telephony-oriented augmentation, and the kriti-telephony methodology pairs roughly 12,000 conversational clips with about 26,000 clean clips. The illustration is deliberately tiny (40 clips) so the arithmetic stays checkable; the same steps scale to tens of thousands.
Step one is the soxi/ffprobe audit the reader rule demands. It flags 30 of 40 clips at 8,000 Hz against the model's 48,000 Hz native rate. Converting durations to samples: the telephony group is 30 × 10 s = 300 s, or 300 × 8,000 = 2,400,000 samples; the studio group is 10 × 10 s = 100 s, or 100 × 48,000 = 4,800,000 samples. Input total: 2,400,000 + 4,800,000 = 7,200,000 samples.
Resampling the 30 telephony clips to 48 kHz is deterministic: 300 s × 48,000 = 14,400,000 samples out, exactly 6× the 2,400,000 in (48,000 ÷ 8,000 = 6). At 16-bit PCM (2 bytes per sample), the telephony group grows from 2,400,000 × 2 = 4.8 MB to 14,400,000 × 2 = 28.8 MB, while the untouched studio group stays at 4,800,000 × 2 = 9.6 MB.
| Clip group | Count | Rate | Duration | Samples | 16-bit size |
|---|---|---|---|---|---|
| Telephony, before | 30 | 8 kHz | 300 s | 2,400,000 | 4.8 MB |
| Telephony, after resample | 30 | 48 kHz | 300 s | 14,400,000 | 28.8 MB |
| Studio, unchanged | 10 | 48 kHz | 100 s | 4,800,000 | 9.6 MB |
Now the two paths. Resample: one sox pass, zero studio time, output rate exactly 48,000 Hz, output length exactly 14,400,000 samples — reproducible in that the same input and settings yield the same 14,400,000-sample output on every run. Re-record: to hit 48 kHz at the source you must re-capture all 300 s (5 minutes) of finished telephony speech, plus session overhead, and only if the original speakers are still available.
For this batch, resampling wins: it is deterministic, adds no studio time, and lands exactly on the model's native rate. The break-even trigger that flips the decision to re-recording is narrow and countable — re-record only if all 30 source speakers are available and the target needs voice content the 8 kHz captures never held; if even one of the 30 is unavailable, re-recording cannot reproduce the batch and resampling wins by default. Confirm the "missing content" half with the convergence check before booking any session.
Decision rules for sample rate mismatches
The first decision is mechanical, not artistic. Run soxi or ffprobe on every reference clip and compare the reported rate against the rate printed on the model card. If the two differ, resample with a sinc-quality filter to the model's native rate before any other processing step — before trimming, before loudness normalization, before the fine-tune job is queued. Framework defaults are the trap here: many training scripts silently accept whatever rate the loader hands them and resample with a fast linear or polyphase kernel chosen for speed, not spectral fidelity. Pin the resampler explicitly in your config and log the output rate of every clip that enters the pipeline.
If the source is telephony-grade and the target is a studio model, resample for evaluation but treat the result as a ceiling, not a fix. Upsampling restores the container rate; it cannot restore the missing band. Use the resampled clip to run your similarity and naturalness checks, then decide whether the residual gap is acceptable for the deployment. If the clone must sound studio-grade, re-record the reference speaker on wideband equipment — that is the only path that adds information rather than stretching it.
If your deployment target is itself telephony, invert the rule deliberately. Fine-tune on telephony-domain audio rather than resampling it up to a studio rate and hoping the model generalizes back down. Botnoi Group's 2026 technical report on domain-adaptive ASR for telephony AI agents documents this pattern: a telephony-oriented fine-tuning set built from live voicebot recordings plus telephony-style augmentation, with character error rate on their telephony data dropping from 23.31% to 9.04% after adaptation. The same logic applies to cloning — match the training domain to the inference domain, and skip the upsampling step entirely.
The five rules, stated once:
| Condition | Action |
|---|---|
| Reference rate ≠ model card rate | Resample with a sinc filter to the model's native rate first; never trust framework defaults. |
| Telephony source (≤8 kHz), studio model target | Resample for evaluation, expect residual bandwidth loss; re-record if studio-grade output is required. |
| Deployment target is telephony | Fine-tune on telephony-domain audio deliberately; skip upsampling to the studio rate. |
| Mixed-rate reference set | Normalize every clip to one rate before batching; never let the loader pick per-file. |
| Rate unknown or unverifiable | Reject the clip from the fine-tune set until soxi/ffprobe confirms it. |
Apply the table as a gate, not a suggestion. A reference set that fails any row should not enter a fine-tune run until the row is resolved, because the cost of a bad resample is paid at inference time in speaker similarity you cannot recover by adding more epochs.
What to do next
| Step | Action | Why it matters |
|---|---|---|
| 1 | Run soxi or ffprobe on every reference clip in your dataset and record the reported sample rate next to each filename before touching the fine-tune config. | You cannot fix a mismatch you have not measured; the 30% similarity drop starts with an unverified input rate. |
| 2 | Compare each recorded rate against the model's native training rate — for NVIDIA Canary 180M/1B Flash models, the target is 48 kHz. | These models are trained on 48 kHz data, so any 8 kHz telephony clip is out of distribution until corrected. |
| 3 | Resample every mismatched clip with a polyphase/sinc filter to the native rate (e.g., 8 kHz → 48 kHz) rather than accepting the pipeline's default resampler. | Deterministic resampling restores the full-bandwidth features the encoder was trained on; a silent default resampler feeds aliasing-contaminated, bandwidth-limited audio into the reference encoder. |
| 4 | After resampling, verify that bandwidth ≥ 4 kHz content survived in each output clip before it enters the reference set. | Upsampling a band-limited 8 kHz source cannot invent missing high frequencies — confirm what actually survived, since degraded embeddings are the root cause of the similarity loss. |
| 5 | Only after steps 1–4 pass, proceed to fine-tuning or zero-shot cloning with the corrected reference clips. | Feeding 8 kHz clips without resampling causes the 30% similarity drop; the check must match the model's training rate before any fine-tune or zero-shot clone. |
| 6 | Re-run the soxi/ffprobe check on the final training set as a last gate, and reject any clip still at a non-native rate. | Prevents a single missed telephony clip from silently dragging down embedding quality across the whole clone. |
Frequently Asked Questions
What is the exact similarity penalty when feeding 8 kHz telephony audio directly into an NVIDIA Canary 180M/1B Flash model without resampling?
Feeding 8 kHz telephony audio into these 48 kHz-trained models without resampling causes a 30% speaker similarity drop.
Which command should be run on every reference clip to confirm its sample rate before fine-tuning?
Run soxi/ffprobe on every reference clip to verify the sample rate matches the model's native training rate (e.g., 48 kHz studio models).
What is the correct preprocessing step to recover full-bandwidth features from 8 kHz clips for a 48 kHz-trained model?
Apply deterministic resampling from 8 kHz to 48 kHz before fine-tuning to restore the full-bandwidth features the encoder was trained on.
Why does 8 kHz input degrade the reference encoder's embedding quality in a 48 kHz pipeline?
Mismatched sample rates feed aliasing-contaminated, bandwidth-limited audio into the reference encoder, degrading embedding quality by limiting usable speech energy to a narrow spectral range.
Quick answers
| What happens to speaker similarity when 8 kHz audio is fed into a 48 kHz-trained voice cloning model? | Speaker similarity drops by 30% due to aliasing and bandwidth-limited input. |
| Why does mismatched sample rate degrade embedding quality in voice cloning? | Mismatched rates feed aliasing-contaminated, bandwidth-limited audio into the reference encoder, degrading embedding quality. |
| What is the recommended fix for sample rate mismatch before fine-tuning? | Resample every reference clip to the model's native training rate using deterministic resampling (e.g., 8 kHz → 48 kHz). |
| What tool should be used to verify sample rates of reference clips? | Run soxi/ffprobe on every reference clip to verify the sample rate matches the model's training rate (e.g., 48 kHz). |
| Which models require telephony audio (8 kHz) to be resampled to 48 kHz? | NVIDIA Canary 180M/1B Flash models, which are trained on 48 kHz data. |
Also worth reading: Why organizations fail voice cloning: 3-second sample vs 2026 adoption: Why organizations fail voice cloning: · Exploring voice cloning effects on audio file fidelity: Exploring voice cloning effects on · Voice cloning: how much clean reference audio? Verify before you commit: Voice cloning: how much clean