Skip to content

imatrix : handle partial entries#7833

Merged
ggerganov merged 1 commit intomasterfrom
gg/imatrix-partial-data
Jun 9, 2024
Merged

imatrix : handle partial entries#7833
ggerganov merged 1 commit intomasterfrom
gg/imatrix-partial-data

Conversation

@ggerganov
Copy link
Copy Markdown
Member

fix #7816

Print warning messages when imatrix entries have zero counts:

compute_imatrix: tokenizing the input ..
compute_imatrix: tokenization took 22899.3 ms
compute_imatrix: computing over 4918 chunks with batch_size 256
compute_imatrix: 12.00 seconds per pass - ETA 16 hours 23.53 minutes
[1]6.6864,[2]9.1522,[3]9.9931,[4]8.8414,
save_imatrix: entry '             blk.17.ffn_down_exps.weight' has partial data (93.75%) - skipping
save_imatrix: entry '             blk.17.ffn_gate_exps.weight' has partial data (93.75%) - skipping
save_imatrix: entry '               blk.17.ffn_up_exps.weight' has partial data (93.75%) - skipping
save_imatrix: entry '             blk.16.ffn_down_exps.weight' has partial data (73.44%) - skipping
save_imatrix: entry '             blk.16.ffn_gate_exps.weight' has partial data (73.44%) - skipping
save_imatrix: entry '               blk.16.ffn_up_exps.weight' has partial data (73.44%) - skipping
save_imatrix: entry '             blk.15.ffn_down_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '             blk.14.ffn_down_exps.weight' has partial data (60.94%) - skipping
save_imatrix: entry '               blk.14.ffn_up_exps.weight' has partial data (60.94%) - skipping
save_imatrix: entry '              blk.1.ffn_down_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '             blk.13.ffn_gate_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '             blk.13.ffn_down_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '             blk.11.ffn_down_exps.weight' has partial data (59.38%) - skipping
save_imatrix: entry '               blk.11.ffn_up_exps.weight' has partial data (59.38%) - skipping
save_imatrix: entry '             blk.15.ffn_gate_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '               blk.15.ffn_up_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '             blk.14.ffn_gate_exps.weight' has partial data (60.94%) - skipping
save_imatrix: entry '             blk.12.ffn_down_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '             blk.12.ffn_gate_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '               blk.12.ffn_up_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '              blk.3.ffn_down_exps.weight' has partial data (98.44%) - skipping
save_imatrix: entry '                blk.1.ffn_up_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '               blk.13.ffn_up_exps.weight' has partial data (62.50%) - skipping
save_imatrix: entry '              blk.1.ffn_gate_exps.weight' has partial data (78.12%) - skipping
save_imatrix: entry '              blk.3.ffn_gate_exps.weight' has partial data (98.44%) - skipping
save_imatrix: entry '                blk.3.ffn_up_exps.weight' has partial data (98.44%) - skipping
save_imatrix: entry '              blk.2.ffn_down_exps.weight' has partial data (96.88%) - skipping
save_imatrix: entry '              blk.2.ffn_gate_exps.weight' has partial data (96.88%) - skipping
save_imatrix: entry '             blk.11.ffn_gate_exps.weight' has partial data (59.38%) - skipping
save_imatrix: entry '                blk.2.ffn_up_exps.weight' has partial data (96.88%) - skipping
save_imatrix: warning: storing only 306 out of 336 entries

save_imatrix: stored collected data after 10 chunks in imatrix.dat

Such entries are not stored in the output matrix to prevent errors when using the imatrix. To prevent this from happening, provide larger and more diverse training data

@ggerganov ggerganov force-pushed the gg/imatrix-partial-data branch from 175a179 to 5a21852 Compare June 8, 2024 09:40
@ggerganov ggerganov merged commit e95beeb into master Jun 9, 2024
@ggerganov ggerganov deleted the gg/imatrix-partial-data branch June 9, 2024 17:19
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: QWEN2 MoE imatrix contains nan's after generating it

1 participant