common/gemma4 : handle parsing edge cases (#21760)#27
Merged
InfernalDread merged 1 commit intoInfernalDread:turboquant_kv_cache_updated_v5from Apr 14, 2026
Merged
Conversation
InfernalDread
merged commit 9af22fc
into
InfernalDread:turboquant_kv_cache_updated_v5
84 of 144 checks passed
InfernalDread
pushed a commit
that referenced
this pull request
Apr 23, 2026
Full investigation log with all tests, results, and the root cause. Upstream TurboQuant activity tracked in #27. Co-Authored-By: tturney@psyguard.ai Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
InfernalDread
pushed a commit
that referenced
this pull request
Apr 23, 2026
Add full asymmetric K/V quantization support for Metal flash attention:
- Pipeline naming uses k{type}_v{type} format for all FA kernels (335 total),
eliminating underscore ambiguity in type names
- 90 turbo × turbo asymmetric instantiations (turbo2/3/4 all combinations)
- 150 q8_0 × turbo asymmetric instantiations (both directions, all head dims)
- Gatekeeper and assertion updated to allow turbo × turbo and q8_0 × turbo pairs
- Zero regression on existing symmetric paths (validated across 4 models, 2 machines)
The q8_0 × turbo kernels fix a silent dispatch failure where mixed q8_0-K + turbo-V
configs would NaN (turbo4-V) or fall to undefined paths (turbo3-V). This enables
the asymmetric quality rescue: q8_0-K + turbo-V recovers near-baseline PPL on
low-bit models where symmetric turbo-K degrades.
Validated on Metal (M2 Pro + M5 Max):
- phi-4-Q8_0: symmetric turbo3 +4.2%, turbo4 +1.7% (no regression)
- Qwen2.5-7B Q4_K_M: q8_0-K + turbo4-V +1.0%, q8_0-K + turbo3-V +2.0% (rescued)
- Qwen3.5-35B MoE, 27B Dense, Mistral-24B: all healthy (no regression)
- Cross-hardware M2/M5 parity confirmed on all tested configs
Closes #27
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.