turboquant_kv_cache_updated_v16#41

Merged

InfernalDread merged 175 commits intoInfernalDread:turboquant_kv_cache_updated_v16from

Addy-ad:addyad-latest

Apr 29, 2026

Commits on Mar 26, 2026

Commits on Mar 27, 2026

perf: parallel k_set_rows_turbo3 + optimise KQ/V dequant — +31% decode (71→94 t/s, 98.5% of F16)

signalnine
and
claude
committed
experiment: batched byte extraction + explicit bit field pre-extract

committed
experiment: profiling modes for turbo3 decode bottleneck isolation

TheTom
and
claude
committed
fix: turbo4 SET_ROWS corruption, tail-block truncation, constant coupling (Issue #29)

seanrasch
and
claude
committed
experiment: 4-entry magnitude LUT + branchless sign (XOR trick)
TheTom
committed
experiment: force non-vec FA path for turbo3 (nl=2 vs nl=8)

TheTom
and
claude
committed
fix: stack overflow in turbo4 CPU init — 64KB array on worker thread stack

seanrasch
and
claude
committed
experiment: zero-LUT select chain — 2-level ternary, no constant memory
TheTom
committed
feat: auto-detect hardware, use 4-mag LUT on pre-M5 (+38-45% decode)

committed
experiment: 2-pair half2 LUT — only 2 constant addresses per lookup
TheTom
committed
experiment: deferred norm multiply (batch float4 * norm at end)
TheTom
committed
revert to proven 4-mag + per-element norm (deferred norm was slower)

TheTom
and
claude
committed
experiment: named-register centroid×norm — 4 constant reads upfront, zero divergence, ternary select
TheTom
committed
revert to 4-mag LUT (proven best), document all findings
TheTom
committed
experiment: inline block processing — bypass template dequant in FA inner loop
TheTom
committed
experiment: inline block WORSE on M2 (-10-15%), reverted to 4-mag

TheTom
and
claude
committed
experiment: FULLY BRANCHLESS FMA decode — zero ternary, zero memory, zero branch
TheTom
committed
final: 12 approaches tested, 4-mag LUT is the hardware limit

TheTom
and
claude
committed
experiment: SIMD SHUFFLE magnitude select — cross-lane LUT replacement
TheTom
committed
experiment: simd_shuffle 14.7 at 8K — close to 4-mag (15.1) but not better

TheTom
and
claude
committed
experiment: fused block dot — per-centroid Q accumulation, 4 constant reads total
TheTom
committed
experiment: fused block dot 8.1 at 8K — worst result, 64 comparisons kill it

TheTom
and
claude
committed
experiment: 4-mag helps M5 at 16K (+2.4%) but hurts at 32K (-7.3%)

TheTom
and
claude
committed
experiment: M5 LUT cost grows to 34% at 32K context

TheTom
and
claude
committed
feat: sparse V dequant — +12% decode at 32K, zero quality loss

TheTom
and
claude
committed
feat: sparse V dequant — +22% decode at 32K on M5, auto-enabled

TheTom
and
claude
committed
Merge remote-tracking branch 'upstream/feature/turboquant-kv-cache' into feature/turboquant-kv-cache
seanrasch
committed
fix: VEC flash-attn Q/K stride mismatch in vec_dot_fattn_vec_KQ_turbo3_0

signalnine
and
claude
committed
Merge pull request #4 from seanrasch/feature/turboquant-kv-cache
TheTom
authored
Revert "Merge pull request #4 from seanrasch/feature/turboquant-kv-cache"
TheTom
committed

Commits on Mar 28, 2026

Commits on Mar 29, 2026

Commits on Mar 31, 2026

Commits on Apr 9, 2026

Commits on Apr 11, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 12, 2026

Commits on Apr 13, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 14, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 15, 2026

Commits on Apr 16, 2026

Commits on Apr 17, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 18, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 19, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 21, 2026

Commits on Apr 22, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 23, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 25, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 26, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed

Commits on Apr 27, 2026

Merge remote-tracking branch 'upstream/master' into addyad-latest
Addy-ad
committed