Skip to content

Layer wise changes for kimi model#951

Closed
abhishek-singh591 wants to merge 58 commits intoquic:mainfrom
abhishek-singh591:layer_wise_kimi
Closed

Layer wise changes for kimi model#951
abhishek-singh591 wants to merge 58 commits intoquic:mainfrom
abhishek-singh591:layer_wise_kimi

Conversation

@abhishek-singh591
Copy link
Copy Markdown
Contributor

@abhishek-singh591 abhishek-singh591 commented Apr 28, 2026

No description provided.

ochougul and others added 30 commits April 13, 2026 05:29
@ochougul @mamtsing
added modeling for kimik2
d37ef9d 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul @mamtsing
able to run kimi model, need to check accuracy
426eed1 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul @mamtsing
bugfix
f3de723 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul @mamtsing
experimentation branch commit
a3abf05 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul @mamtsing
added MLA with/WO fusion, the caching for different config needs to b…
a27e272 
…e sorted

Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@mamtsing
Add prefill only moe changes from kimik2 branch
4e025c6 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
Change Cache for compressed KV and k_rope
2c5358d 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
fix dynamic axis and output mismatch
30c57e9 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@quic-mamta @mamtsing
Split kv_a_proj_with_mqa weights to get ckv and k_pe
0517d93 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
update min_masked_attention_value, head blocking, attn in fused_forwa…
1398388 
…rd, prefillonly transform

Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
Add replicatekvhead transform
8aaa56a 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
Add KV blocking and update head blocking
b05ca6e 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
Update modeling_deepseek_qeff.py
2e44321 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
update kv heads in example_inputs during export and kv_blocking
8c99ee0 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
Add example script
2859c19 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
update fused forward for loops
13da8ef 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
fix reshapes in fused_forward
9e432dc 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
write_only and read_onnly for blocked_kv
b625113 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@ochougul @mamtsing
removed commented code
768ce85 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@mamtsing
update compressed-tensors and tiktoken version
4f11f58 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
add all forward for h and kv blocking
4cbb376 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
removing blocking config changes
04fa702 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@ochougul @mamtsing
fixed basic forward
900ee0b 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul @mamtsing
fixed for kimi k2
f763671 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@mamtsing
fix lint format
dc20495 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
remove redundencies
877a2de 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
fix no absorption for head blocking
38910d3 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
Add head blocking in full pkv forward
a0f0e61 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@quic-mamta
Merge branch 'main' into mla_perf
0965b5e
@mamtsing
fix tests
5b56df4 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
quic-mamta and others added 26 commits April 15, 2026 09:42
@quic-mamta @mamtsing
Merge branch 'main' into mla_perf
91526ab 
Signed-off-by: Mamta Singh <168400541+quic-mamta@users.noreply.github.com>
@mamtsing
added orig forward from snippet
37dd1bb 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
add support for subfunctions
210d953 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@quic-mamta
Merge branch 'main' into mla_perf
09fd219
@mamtsing
use generalized kv blocking
6b04a57 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
use generalised infra for h_blocking with ckv and full kv
2126672 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@quic-mamta
Merge branch 'main' into mla_perf
6d070ac
@mamtsing
remove redundant files
c9fd13e 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
remove env variables
5c23f53 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@quic-mamta
Merge branch 'main' into mla_perf
d05bbf8
@mamtsing
simplify mla_absorption_config
8e7ddbe 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
address review comments
c966cc9 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@mamtsing
fix mla_absorption_config
01867ea 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
@ochougul
added head expansion for kv blocking
4f7d28c 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul
Merge branch 'mla_perf' of github.com:quic/efficient-transformers int…
fb20758 
…o mla_perf
@ochougul
fixed kv with kv replication
b05aa06 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@ochougul
minor fix
c24d851 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
@abhishek-singh591
Added all changes of layer wise for kimi model
bdbf50c 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Made minor fix
4c069c3 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Update run.py
bf1b9e3 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Update modeling_qeff.py
ab312d0 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Made minor fix
7cd93e3 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Made minor fix
067ce7d 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Made minor fix
30c8e24 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Made minor fix
b211a9a 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
Added thread pool for loading the QPC
106f065 
Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@abhishek-singh591 abhishek-singh591 mentioned this pull request Apr 29, 2026
Closed
@anujgupt-github
Copy link
Copy Markdown
Contributor

anujgupt-github commented Apr 30, 2026

@abhishek-singh591 this needs a cleanup. Quite many unnecessary changes have come in, perhaps a rebase will be good.

@abhishek-singh591
Copy link
Copy Markdown
Contributor Author

abhishek-singh591 commented Apr 30, 2026 •
edited
Loading

@abhishek-singh591 this needs a cleanup. Quite many unnecessary changes have come in, perhaps a rebase will be good.

I already had raised a new PR #954 directly over mainline, this was over mla_perf branch, Hence i am closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@abhishek-singh591 @anujgupt-github @ochougul @mamtsing @quic-mamta