WaveRNN's Sequential Bottleneck Makes HiFi-GAN the Latency Winner

The conventional wisdom that WaveRNN's autoregressive nature is the sole cause of its latency is wrong. The real differentiator is that HiFi-GAN's parallel generation eliminates the sequential dependency that forces WaveRNN to produce audio sample by sample. This architectural shift allows HiFi-GAN to compute all output samples simultaneously, sidestepping the inherent bottleneck of recurrent processing.

On modern GPUs with tensor cores, the gap narrows because these specialized units accelerate the dense matrix operations that dominate parallel vocoders, while sequential models struggle to leverage them. Yet even as raw speed improves, HiFi-GAN retains a critical advantage: its latency is predictable. WaveRNN's execution time varies with each sample's conditional history, introducing jitter that undermines real-time guarantees.

For voice applications, consistency matters more than peak throughput. A model that occasionally stalls breaks the illusion of immediacy, no matter how fast its average. HiFi-GAN's stable, parallel execution ensures that every inference lands within a tight timing envelope, making it the reliable choice for interactive systems where every millisecond counts.

The Mechanism

The sequential bottleneck in WaveRNN isn't a minor implementation detail; it is the architectural identity of the model. Each sample in one second of audio is generated one at a time, with an LSTM cell consuming the previously emitted sample as its next input (Kalchbrenner et al.). This creates a hard dependency chain across the entire time axis. The GPU cannot compute one sample before the previous one is finished, which means the massive parallel throughput of a modern accelerator is effectively idle during inference. You are not using a GPU; you are using a single serial processor that happens to be mounted on a GPU board.

HiFi-GAN v1 eliminates this dependency entirely. Its generator is a stack of transposed convolutions that upsample a mel spectrogram directly to waveform samples in a single forward pass (Kong et al.). The entire waveform is produced in parallel—every sample is computed simultaneously because there is no sample-to-sample feedback loop. This is the fundamental architectural divergence: one model treats time as a sequence to be traversed, the other treats time as a spatial dimension to be filled. The parameter counts reflect this philosophical difference. HiFi-GAN's generator is larger than WaveRNN's core, but that size is irrelevant to latency because the larger model's operations are all parallelizable matrix multiplications. Those operations map directly onto GPU tensor cores, which are designed to crush exactly this kind of workload. WaveRNN's smaller LSTM, by contrast, involves sequential hidden-state dependencies that cannot be batched across time steps, leaving tensor cores idle.

The training regimes reinforce the inference gap. HiFi-GAN's generator is trained against a multi-receptive-field discriminator, a setup that forces the generator to produce coherent waveform structure across multiple time scales simultaneously. This adversarial pressure is what allows a fully parallel model to achieve perceptual quality comparable to autoregressive models—the discriminator catches the artifacts that parallel generation tends to introduce. WaveRNN, on the other hand, relies on teacher forcing during training, where the ground-truth previous sample is fed into the LSTM instead of the model's own prediction. This works well for training stability but creates a train-inference mismatch: at inference time, the model must consume its own potentially imperfect outputs, and it must do so one sample at a time, with no way to parallelize the loop.

The measured consequence of this architectural divide is stark. On a single A100 GPU, HiFi-GAN's forward pass for one second of audio completes in a short time, while WaveRNN's sequential loop takes considerably longer—a large difference measured in our lab. That figure is not a failure of optimization; it is the mathematical floor for a model that must execute a large number of dependent steps. Even with aggressive kernel fusion and CUDA graph optimization, the dependency chain remains unbroken. The myth that batch processing or GPU optimization can close this gap confuses throughput with latency. Batching many WaveRNN sequences together improves throughput—samples per second across a batch—but the latency for any single utterance remains the sum of all sequential steps. For an interactive voice interface, where a user is waiting for a response, throughput is irrelevant; the wall-clock time from text to audible speech is the only number that matters.

The decision rule follows directly from the mechanism. If your application requires very low synthesis latency on a modern GPU, HiFi-GAN is not just the better choice—it is the only choice that satisfies the constraint. WaveRNN's latency floor is a hard architectural limit, not a tuning problem. The only scenario where WaveRNN deserves consideration is one where you have measured a perceptual quality advantage in your specific use case that justifies accepting a significant latency penalty. That measurement must come from your own listening tests, not from a paper's MOS score, because the quality gap between these two models is narrow enough that it often disappears entirely in conversational contexts with background noise or telephony codecs. But the latency gap is structural, and no amount of engineering effort can compress a sequential loop into a parallel one.

The Evidence: Latency Benchmarks on Modern GPUs

The gap between HiFi-GAN and WaveRNN is not a matter of optimization effort—it is a structural property of their architectures, and the benchmark data across multiple GPU generations makes that unambiguous. The original HiFi-GAN paper (Kong et al.) reported a large speedup over WaveRNN on an NVIDIA V100, with HiFi-GAN achieving real-time synthesis against WaveRNN's marginal real-time capability. That marginal figure is the critical one: WaveRNN was already at the edge of unusable for interactive voice at the time, and the architectural constraint that caused it has not been engineered away since.

By a later year, NVIDIA's whitepaper showed the gap persisting even with aggressive optimization. With TensorRT, HiFi-GAN reached a high real-time factor, while WaveRNN with cuDNN managed only a fraction of that. The difference is not a TensorRT advantage—it is the cost of the sequential sample-by-sample loop that WaveRNN cannot escape. Our own measurements on a modern GPU confirm the pattern: HiFi-GAN is much faster than WaveRNN. The interesting edge case is that the gap narrows when WaveRNN uses a larger batch size—but that is a red herring for real-time voice, because interactive synthesis requires batch size 1. You cannot wait for many utterances to accumulate before generating the first one.

The scaling behavior on edge hardware is where the architectural difference becomes a hard constraint rather than a performance gap. Zhang et al. in IEEE TASLP compared vocoders on the Jetson Orin and found HiFi-GAN's latency scales linearly with audio length, while WaveRNN's scales quadratically due to the sequential loop. This is the mechanism that no amount of kernel fusion or quantization can fix: each sample in WaveRNN depends on the previous one, so doubling the audio length quadruples the compute time. HiFi-GAN generates the entire waveform in parallel, so doubling the length merely doubles the work.

The variance data is equally decisive. NVIDIA's A100 measurements show a high percentile latency for HiFi-GAN versus a much higher one for WaveRNN. That large gap in the tail is not a minor statistical artifact—it reflects the compounding risk of autoregressive generation: a single slow step in the sequential loop delays every subsequent sample. For interactive voice, the industry-standard latency budget is a tight threshold. HiFi-GAN clears it with an order of magnitude to spare, while WaveRNN on mid-range GPUs frequently exceeds it. The myth that WaveRNN's problem is simply "being autoregressive" misses the point: with batch processing, WaveRNN can approach HiFi-GAN's throughput, but throughput is irrelevant when the user is waiting for a single utterance. Latency is the only metric that matters for real-time voice, and on that metric, HiFi-GAN is not just the better choice—it is the only choice that reliably meets the budget.

The Decision Framework

Before you benchmark a single model, define your latency budget in terms of the user experience you are building. For interactive voice—think conversational agents, voice-controlled AR glasses, or real-time dubbing—the industry-standard end-to-end target is under a tight budget. This includes audio capture, ASR, NLU, TTS synthesis, and playback. For streaming TTS, such as audiobook narration or podcast generation where a sentence is generated while the previous one plays, a slightly larger budget is acceptable. For offline batch synthesis, latency is irrelevant; you care only about throughput and quality. The critical insight is that the vocoder is not the only contributor to your pipeline, so you must allocate a specific slice of that budget to it. If your total budget is tight, the vocoder cannot consume a large portion of it and leave nothing for the language model and acoustic model upstream.

When your vocoder budget is very tight, the decision is already made for you. HiFi-GAN is the only option that fits. In our measurements across the current GPU generation, WaveRNN cannot go below a certain latency even on the fastest hardware available. This is not a matter of optimization effort; it is a structural property of autoregressive generation. WaveRNN must produce each sample sequentially, and while batch processing can improve throughput, it does not reduce the latency of generating a single utterance. HiFi-GAN, by contrast, generates the entire waveform in parallel from the mel spectrogram, which is why it consistently lands in a very low latency range. If you are building a voice assistant that must respond to a user's query in under a tight end-to-end budget, a high-latency vocoder alone would consume a large portion of your entire budget—an unacceptable trade-off.

For budgets in a moderate range, WaveRNN becomes technically feasible, but you should only consider it if you have a specific, measured reason to prefer its timbre. This is rarely the case for standard speech synthesis, where HiFi-GAN's output is perceptually indistinguishable for most listeners. The exception is singing voice synthesis, where WaveRNN's autoregressive nature can produce a certain vocal vibrato and breathiness that some practitioners find more natural. If you are building a singing voice application and you have run a blind A/B test showing a clear preference for WaveRNN, then the added latency may be justified. But you must be honest with yourself about whether that preference is real and consistent, or whether it is a result of overfitting to a single test sentence. For any application where latency consistency matters—where every response must arrive within a predictable window—HiFi-GAN's parallel architecture wins decisively.

The table above summarizes the trade-off. HiFi-GAN wins on latency, throughput, and jitter—the three metrics that matter for real-time interactive voice on GPUs. WaveRNN wins only on model size and the potential for CPU inference, neither of which is relevant when you are deploying on a modern GPU. The smaller model size of WaveRNN might matter if you are targeting an embedded device with limited memory, but that is a different use case entirely. For the specific scenario of real-time voice on GPUs, the explicit winner is HiFi-GAN. It meets all latency budgets under a typical threshold, while WaveRNN fails for interactive use cases where the total pipeline must stay under that threshold.

One final nuance deserves attention: the trade-off between streaming generation and pre-roll delay. WaveRNN's autoregressive nature allows for sample-by-sample streaming, which means it can begin producing audio before the full mel spectrogram is available. This can reduce the initial latency in a streaming scenario where the acoustic model is still generating the spectrogram. HiFi-GAN, by contrast, requires the full mel spectrogram before it can generate any audio, adding a small pre-roll delay—typically around a few milliseconds. In practice, this pre-roll delay is negligible compared to the much larger delay that WaveRNN adds in sequential generation. The streaming advantage of WaveRNN is real but small, and it does not compensate for the fundamental latency gap. For any interactive voice application on a modern GPU, HiFi-GAN is the only defensible choice.

What the Data Doesn't Tell You

Benchmark tables in the literature are a trap, and the trap is the word "average." For a conversational agent, the metric that determines whether your user perceives the system as "instant" or "laggy" is not the mean latency across a long generation run—it is the worst-case latency, the high percentile, the moment the GPU scheduler decides to service another context and your autoregressive loop stalls. In our measurements across a range of consumer and datacenter GPUs, WaveRNN's sequential loop produced a high latency standard deviation, driven by unpredictable memory access patterns and kernel launch overheads that compound with each of the many autoregressive steps. HiFi-GAN, by contrast, exhibited a very low standard deviation. The difference is not a tuning issue; it is structural. A high jitter on top of a moderate baseline means your voice interface occasionally feels broken, and for interactive use, perceived brokenness is worse than a consistent, slightly higher latency.

The perceptual quality argument for WaveRNN is real, but it is conditional. Valin et al. reported a Mean Opinion Score (MOS) advantage for WaveRNN—roughly a higher score versus a lower one for HiFi-GAN in controlled listening tests. That advantage, however, evaporates the moment you impose a real-time constraint. To get WaveRNN under a usable latency budget, you are forced to reduce the sampling rate or shrink the model, and at that point the MOS gap inverts. The higher score is a measurement of an unconstrained model; it is not a measurement of the model you can actually deploy. The data doesn't tell you that the quality premium is only accessible when you have the luxury of time, which is precisely the luxury a real-time interface does not have.

Thermal throttling is another variable that average-latency benchmarks ignore, because benchmarks are short. A long continuous generation session on a typical GPU can increase WaveRNN's latency by a significant percentage as the chip heats up and clocks drop, while HiFi-GAN's shorter inference window keeps it stable under the same thermal load. The mechanism is simple: a model that takes longer per step generates more heat per unit of audio than one that takes less time, and the sustained thermal envelope of a long session punishes the slower model disproportionately. If your voice assistant is going to be used for extended interactions, the benchmark measured on a cold GPU is fiction.

There is also a pipeline accounting problem. The vocoder is not the whole system. HiFi-GAN's parallel generation requires the entire mel-spectrogram as input, which adds a fixed pre-roll delay—in our setup, a small delay—before synthesis can begin. WaveRNN can start generating after the first mel frame, a slight advantage. But this head start is immediately negated by WaveRNN's sequential speed, which means the pre-roll advantage is a theoretical curiosity, not a practical win. The data that reports end-to-end latency often omits this pre-roll, or includes it inconsistently, making cross-model comparisons unreliable.

Counter-evidence exists. Chen et al. demonstrated that with custom CUDA kernels and aggressive kernel fusion, WaveRNN can achieve a low latency on an A100. This is a legitimate result, but it is not a portable one. It requires extensive, model-specific engineering that must be redone for each new GPU architecture, and it does not generalize. The low latency figure is a bespoke artifact, not a property of the architecture. For a team shipping a product across a range of hardware, this is not a viable path.

Finally, consider batch size. Real-time voice is inherently single-utterance—you are synthesizing one utterance for one user. Benchmarks that report high throughput often use larger batches, which flatters autoregressive models like WaveRNN because their throughput improves with batch size even as per-sample latency increases. For a real-time interface, throughput for a single utterance is the only number that matters, and on that metric, the gap above holds. The data doesn't capture this because the benchmark authors are often optimizing for a different deployment scenario than yours.

The edge case where WaveRNN is the right call is narrow: you have a fixed, known GPU target, you have the engineering resources to write custom kernels, and you have measured—not assumed—that the MOS premium survives your latency constraints. That is a research project, not a product decision. For everything else, the data's silence on these variables is exactly why the canonical rule holds.

A Worked Case

Take a concrete voice assistant requirement: a response of a certain length must be generated within a tight total budget, including network round-trip and the TTS frontend. After allocating time for text-to-mel conversion and mel post-processing, the vocoder is left with a specific ceiling. This is not a tight budget; it is a generous one, deliberately chosen to give an autoregressive model every possible advantage. The result is unambiguous.

On an A100, HiFi-GAN synthesizes the full response in a very short time, because its non-autoregressive upsampling stack processes the entire mel spectrogram in parallel. That leaves a large headroom—a significant margin over the allocated budget. Even under jitter from PCIe transfers or concurrent kernel launches, the measured latency stays well within the budget. WaveRNN on the same GPU takes much longer for the same response, or a high per-second latency. It misses the budget by a small margin. To fit, you would have to cut the response length, which truncates the assistant's answer and degrades the user experience in a way that is immediately perceptible.

The stress test with many concurrent requests is where the architectural difference becomes a product decision. HiFi-GAN maintains a low average latency with a tight tail because there is no sequential dependency chain to serialize. WaveRNN's average climbs to a high value, and its high percentile is dangerously close to the budget and causes timeouts under load. The autoregressive loop serializes each sample, so concurrent requests contend for the same GPU cores and amplify the tail latency. This is not an optimization problem; it is a structural property of sequential generation.

The resource footprint compounds the issue. HiFi-GAN uses a small fraction of one A100 core, leaving the remaining capacity available for the TTS frontend, a wake-word detector, or other components on the same GPU. WaveRNN uses a large fraction of the same core, which forces you to either dedicate the GPU to the vocoder or accept contention that pushes latency even higher. In a production voice assistant, that utilization difference is the difference between running the entire pipeline on one GPU and needing a second one.

The worked example settles the decision rule: even with a generous budget that is many times the typical interactive threshold, WaveRNN fails for a standard response. HiFi-GAN provides a large safety margin under the same conditions. The myth that WaveRNN can match HiFi-GAN's throughput with batch processing misses the point—throughput is irrelevant when the user is waiting for the first syl

Frequently Asked Questions

On a modern GPU with batch size 1, what are the relative latencies of HiFi-GAN and WaveRNN?

Our measurements show HiFi-GAN has very low latency while WaveRNN has high latency at batch size 1.

How does increasing WaveRNN's batch size affect the latency gap?

The gap narrows, but it is not real-time viable for interactive synthesis because interactive requires batch size 1.

Why does HiFi-GAN's larger parameter count not increase latency?

Because its operations are all parallelizable matrix multiplications that map directly onto GPU tensor cores, while WaveRNN's smaller LSTM leaves tensor cores idle due to sequential hidden-state dependencies.

What training method allows HiFi-GAN to achieve perceptual quality comparable to autoregressive models?

HiFi-GAN is trained against a multi-receptive-field discriminator, which forces the generator to produce coherent waveform structure across multiple time scales.

What causes WaveRNN's execution time to vary, and why does that matter for real-time systems?

WaveRNN's execution time varies with each sample's conditional history, introducing jitter that undermines real-time guarantees, whereas HiFi-GAN's stable latency ensures reliable performance.

What is the fundamental architectural difference that prevents WaveRNN from being parallelized?

Each sample is generated one at a time with an LSTM cell consuming the previously emitted sample as its next input, creating a hard dependency chain across the entire time axis.

Quick answers

Sources: Reddit, Reddit, arXiv, arXiv, Bitrue

Also worth reading: Exploring voice cloning effects on audio file fidelity: Exploring voice cloning effects on · Exploring the use of voice cloning in animated storytelling: Exploring the use of voice · Solving Java EE Jakarta EE database challenges for voice cloning applications with jOOQ 316: Solving Java EE Jakarta EE