Skip to content

Windows fixes#31

Merged
ggerganov merged 5 commits intoggml-org:masterfrom
etra0:windows-fixes
Mar 12, 2023
Merged

Windows fixes#31
ggerganov merged 5 commits intoggml-org:masterfrom
etra0:windows-fixes

Conversation

@etra0
Copy link
Copy Markdown
Contributor

@etra0 etra0 commented Mar 12, 2023 •
edited
Loading

This would be the initial PR to be able to compile stuff in Windows.

In particular, MSVC is very picky about the features you can use and you cannot.

With C++11

  • You cannot use designated initializers (when initializing a struct, you cannot specify the fields names)
  • You cannot use VLAs, so I changed it to a vector.

A PR for the CMake build system (as agreed in #22) will be separated.

These changes were tested with MSVC 19.34.31937.0 (VS Studio 2022) and in macOS 12.6 with Apple clang version 13.1.6

@NotNite NotNite mentioned this pull request Mar 12, 2023
Closed
@etra0 etra0 marked this pull request as draft March 12, 2023 04:50
@etra0 etra0 marked this pull request as ready for review March 12, 2023 05:18
@0xbitches 0xbitches mentioned this pull request Mar 12, 2023
Closed
ggerganov
ggerganov requested changes Mar 12, 2023
View reviewed changes
Comment thread utils.cpp Outdated
@ggerganov ggerganov mentioned this pull request Mar 12, 2023
Closed
@etra0 etra0 requested a review from ggerganov March 12, 2023 15:47
@ggerganov ggerganov merged commit eb062bb into ggml-org:master Mar 12, 2023
@NouamaneTazi NouamaneTazi mentioned this pull request Mar 16, 2023
Open
Hades32 pushed a commit to Hades32/llama.cpp that referenced this pull request Mar 21, 2023
@antimatter15
Merge pull request ggml-org#31 from anzz1/ci_test
7b24407 
CI fine tuning
@NouamaneTazi NouamaneTazi mentioned this pull request Mar 22, 2023
Open
@Bearsaerker Bearsaerker mentioned this pull request Mar 12, 2025
Closed
@graehl graehl mentioned this pull request Jul 8, 2025
Merged
theo77186 pushed a commit to theo77186/llama.cpp that referenced this pull request Oct 28, 2025
@ngxson
Merge pull request ggml-org#31 from rujialiu/mrope-fix
ef1af68 
Mrope fix
jesusmb1995 pushed a commit to jesusmb1995/llama.cpp that referenced this pull request Oct 30, 2025
@olyasir
Merge pull request ggml-org#31 from ogad-tether/temp-latest
87f40dd 
implement logging redirection
@sainnhe sainnhe mentioned this pull request Jan 25, 2026
Closed
rururush pushed a commit to USTC-ADSL/llama.cpp that referenced this pull request Mar 16, 2026
@chraac
bug: fix benchmark debug warning (ggml-org#31)
8b652dd 
* print build type

* wip

* print compiling flags

* wip

* wip
TheTom referenced this pull request in TheTom/llama-cpp-turboquant Mar 25, 2026
@TheTom @claude
docs: bisect confirms block size innocent, rotation access is the bug #…
4742a3b 
…31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TheTom referenced this pull request in TheTom/llama-cpp-turboquant Mar 25, 2026
@TheTom @claude
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) #31 …
1f02172 
…#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TheTom referenced this pull request in TheTom/llama-cpp-turboquant Mar 26, 2026
@TheTom @claude
docs: bisect confirms block size innocent, rotation access is the bug #…
3d9bcc7 
…31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TheTom referenced this pull request in TheTom/llama-cpp-turboquant Mar 26, 2026
@TheTom @claude
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) #31 …
cf6270c 
…#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
spiritbuun referenced this pull request in spiritbuun/buun-llama-cpp Mar 27, 2026
@spiritbuun @claude
docs: benchmark results for turbo4 fix, sparse V, multi-model validation
271deea 
- turbo4 K+V results on Qwen3.5-27B (-0.32% vs q8_0) and Qwen3-14B (+6.3%)
- Sparse V dequant benchmarks: MoE native dequant +10.9% at 8K
- Gemma-3 turbo3 results post-iSWA fix (+3.3%)
- KVLinC no-K-rotation negative result
- Speculative decoding negative result
- CUDA 13.2 compatibility verified
- Experiments #31, TheTom#39, TheTom#42, TheTom#45, TheTom#49, TheTom#50, TheTom#51 status updates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
didlawowo pushed a commit to didlawowo/llama.cpp that referenced this pull request Mar 27, 2026
@TheTom @claude
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) gg…
04c8f5e 
…ml-org#31 ggml-org#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
InfernalDread referenced this pull request in InfernalDread/llama.cpp Apr 4, 2026
@TheTom @claude
docs: bisect confirms block size innocent, rotation access is the bug #…
e20ce5d 
…31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
InfernalDread referenced this pull request in InfernalDread/llama.cpp Apr 4, 2026
@TheTom @claude
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) #31 …
c52aef5 
…#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
didlawowo pushed a commit to didlawowo/llama.cpp that referenced this pull request Apr 4, 2026
@TheTom @claude @didlawowo
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) gg…
fb758cc 
…ml-org#31 ggml-org#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
itme-brain pushed a commit to itme-brain/llama.cpp that referenced this pull request Apr 16, 2026
@TheTom @claude @itme-brain
docs: bisect confirms block size innocent, rotation access is the bug g…
66a9b60 
…gml-org#31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
itme-brain pushed a commit to itme-brain/llama.cpp that referenced this pull request Apr 16, 2026
@TheTom @claude @itme-brain
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) gg…
b866047 
…ml-org#31 ggml-org#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
erazortt pushed a commit to erazortt/llama.cpp that referenced this pull request Apr 17, 2026
@TheTom @claude
docs: bisect confirms block size innocent, rotation access is the bug g…
b0234bd 
…gml-org#31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
erazortt pushed a commit to erazortt/llama.cpp that referenced this pull request Apr 17, 2026
@TheTom @claude
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) gg…
ebe0a23 
…ml-org#31 ggml-org#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ausshir pushed a commit to ausshir/llama.cpp-iso-rocm that referenced this pull request Apr 20, 2026
@TheTom @claude @ausshir
docs: bisect confirms block size innocent, rotation access is the bug g…
96f847b 
…gml-org#31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ausshir pushed a commit to ausshir/llama.cpp-iso-rocm that referenced this pull request Apr 20, 2026
@TheTom @claude @ausshir
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) gg…
69dbad1 
…ml-org#31 ggml-org#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
YuruDeveloper pushed a commit to YuruDeveloper/llama.cpp-quant that referenced this pull request Apr 21, 2026
@TheTom @claude @YuruDeveloper
docs: bisect confirms block size innocent, rotation access is the bug g…
60ddbb5 
…gml-org#31

Block 128: PPL=165.6 (same as block 32)
Disabled Q rotation: PPL=165.6 (same)
Root cause: dynamic_cast fails for MoE hybrid memory context.
Q rotation and V inverse rotation never execute.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
YuruDeveloper pushed a commit to YuruDeveloper/llama.cpp-quant that referenced this pull request Apr 21, 2026
@TheTom @claude @YuruDeveloper
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) gg…
5839887 
…ml-org#31 ggml-org#30

ROOT CAUSE: pre-rotate-queries never executed because:
1. Q ne[0]=256 (GQA concatenated heads), rotation matrix ne[0]=128
2. mctx dynamic_cast failed for MoE hybrid memory

FIX: put inverse WHT rotation back in dequantize_full_block.
This is slower (10.7 tok/s vs 77.7) but produces CORRECT results.

PERPLEXITY RESULTS:
- f16:     6.121
- q8_0:    6.111
- q4_0:    6.142
- turbo3:  6.194 (+1.2% vs q8_0) ✅

The speed optimization (pre-rotate-queries) needs to be reimplemented
to work with GQA head layout and hybrid memory types.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@etra0
Windows fixes (ggml-org#31)
fde84af 
* Apply fixes suggested to build on windows

Issue: ggml-org#22

* Remove unsupported VLAs

* MSVC: Remove features that are only available on MSVC C++20.

* Fix zero initialization of the other fields.

* Change the use of vector for stack allocations.
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
@ikawrakow
Fix build when iqk_mul_mat is disabled (ggml-org#31)
5ff9970 
Ref ggml-org#29

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@etra0 @ggerganov