Skip to content

imatrix : fix wname for mul_mat_id ops#6271

Merged
ggerganov merged 2 commits intomasterfrom
gg/fix-imatrix-mm-id-name
Mar 24, 2024
Merged

imatrix : fix wname for mul_mat_id ops#6271
ggerganov merged 2 commits intomasterfrom
gg/fix-imatrix-mm-id-name

Conversation

@ggerganov
Copy link
Copy Markdown
Member

Fix a regression for computing the imatrix of MoE models introduced in #6083

The wname has to be updated with the expert weight name. Otherwise, master stops early:

Oops: inconsistent size for blk.0.ffn_down.0.weight,  (view) (4096 vs 14336)

Repro:

./imatrix -m ./models/mixtral-8x7b-32k-fast/ggml-model-f16.gguf -f build/wikitext-2-raw/wiki.train.raw

Comment thread examples/imatrix/imatrix.cpp Outdated
@ggerganov ggerganov merged commit a0e584d into master Mar 24, 2024
@ggerganov ggerganov deleted the gg/fix-imatrix-mm-id-name branch March 24, 2024 14:18
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 3, 2024
* imatrix : fix wname for mul_mat_id ops

* also filter tensor names in mul_mat_id ops

---------

Co-authored-by: slaren <slarengh@gmail.com>
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* imatrix : fix wname for mul_mat_id ops

* also filter tensor names in mul_mat_id ops

---------

Co-authored-by: slaren <slarengh@gmail.com>
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
* imatrix : fix wname for mul_mat_id ops

* also filter tensor names in mul_mat_id ops

---------

Co-authored-by: slaren <slarengh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants