perf(qwen3-tts): use LATENCY hint for OV compilations; fix streaming defaults to Qwen upstream

[Conradzz]

Summary

1. Use PERFORMANCE_HINT=LATENCY for all Qwen3-TTS OV compilations

This pipeline is a single-stream autoregressive decode loop at batch=1. Without an explicit hint, the GPU plugin uses PerformanceMode.UNDEFINED, which doesn't optimize for single-stream latency. Setting LATENCY pins one execution stream and minimizes per-infer dispatch overhead. CPU already defaults to LATENCY-like behavior; the hint is set explicitly there for consistency.

Measured impact (B70 / Xe2 / OpenVINO 2024.x GPU plugin):

• Talker decode: ~68–92 ms/frame → ~22 ms/frame (~3–4× speedup)
• Overall first-audio latency for a short phrase: ~5s → ~1.2s

2. Restore streaming defaults to Qwen-recommended values

stream_chunk_frames was 50, which diverges from Qwen's recommended chunk_size=300 (ref). Restored to 300. stream_left_context kept at 25, matching upstream's left_context_size=25.

Test plan

• Qwen3-TTS 1.7B Base voice clone loads cleanly on GPU with LATENCY hint
• Verified on Intel Arc B70 (Battlemage, 32 GB, OpenVINO 2024.x)
• Streaming defaults match upstream Qwen3-TTS reference

[SearchSavior]

@Conradzz Hello, thank you for the PR!

Have you tested longer than a few seconds? Admittedly 50 was an AI mistake I missed; the default value of stream_chunk_frames is 300, which is what qwen reccomends and partly a constraint of the architecture since audio codebooks must be generated in a sequence that depends on the previous set of codebooks. To stream coherent chunks we need enough of them; see here

https://github.com/QwenLM/Qwen3-TTS/blob/022e286b98fbec7e1e916cb940cdf532cd9f488e/qwen_tts/core/tokenizer_12hz/modeling_qwen3_tts_tokenizer_v2.py#L886

For me, your defaults are breaking prosody with frequent pauses, vs waiting for larger chunks to accumulate. However the latency hint is a good addition, and I would be open to merging that.

---

I suggest using the paper for qwen3-tts in your agent's context.

This PR does not demonstrate implementation considerations about what the code was designed to do vs what it is in the current form. Notice how the agent writes Default stream_chunk_frames=50 corresponds to ~4.2s of audio at the 12 Hz codec rate and rushes into details present in the code now, versus asking, why was this default set this way in the first place. Those details' aren't in the OpenArc codebase, and your agent did not have the right context to make good decisions.

Before we go any further, I encourage you to test stream_chunk_frames=50 vs stream_chunk_frames=300 and let me know how each value sounds. Hold stream_left_context=25.

[Conradzz]

You're correct, I found we were running with the wrong parameters a little later on. I'm still working on my development team.

…

On Thu, Apr 9, 2026 at 9:18 PM Emerson Tatelbaum ***@***.***> wrote: *SearchSavior* left a comment (SearchSavior/OpenArc#78) <#78 (comment)> @Conradzz <https://github.com/Conradzz> Hello, thank you for the PR! Have you tested longer than a few seconds? Admittedly 50 was an AI mistake I missed; the default value of stream_chunk_frames is 300, which is what qwen reccomends and partly a constraint of the architecture since audio codebooks must be generated in a sequence that depends on the previous set of codebooks. To stream *coherent* chunks we need enough of them; see here https://github.com/QwenLM/Qwen3-TTS/blob/022e286b98fbec7e1e916cb940cdf532cd9f488e/qwen_tts/core/tokenizer_12hz/modeling_qwen3_tts_tokenizer_v2.py#L886 For me, your defaults are breaking prosody with frequent pauses, vs waiting for larger chunks to accumulate. However the latency hint is a good addition, and I would be open to merging that. ------------------------------ I suggest using the paper for qwen3-tts in your agent's context. This PR does not demonstrate implementation considerations about what the code was designed to do vs what it is in the current form. Notice how the agent writes Default stream_chunk_frames=50 corresponds to ~4.2s of audio at the 12 Hz codec rate and rushes into details present in the code now, versus asking, why was this default set this way in the first place. Those details' aren't in the OpenArc codebase, and your agent did not have the right context to make good decisions. Before we go any further, I encourage you to test stream_chunk_frames=50 vs stream_chunk_frames=300 and let me know how each value sounds. Hold stream_left_context=25. — Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEPZDPNEOIZEAD7SGXNPEL4VBDVDAVCNFSM6AAAAACXTFTUBGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DEMJZGE3DAMRYHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>