Skip to content

feat: add kai&qnn-vl&opencl #489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yirongjie merged 23 commits into from
Oct 27, 2025
Merged

feat: add kai&qnn-vl&opencl #489

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
71d7a68
feat: add qwen2vl-qnn & flashattn2 support (#291)
UbiquitousLearning Jun 9, 2025
8476dd0
Squashed commit of the following:
yirongjie May 27, 2025
6b30afc
feat: add FlashAttention2 && fix: MULTIMODELROPE
yirongjie Jun 7, 2025
0ce66c5
remove broken submodule
Jun 9, 2025
627ea41
fix: support for mac && fix xnn
yirongjie Jun 10, 2025
6aa2881
feat: add showui-cut in vtp;
yirongjie Jun 11, 2025
60da3be
feat: add klaidiai linear op
yirongjie Jun 13, 2025
ec1db94
fear: add qwen2-vl-7b
yirongjie Jun 14, 2025
42e5bb7
feat: add i8mm kleidai for a32bi4
yirongjie Jun 14, 2025
012bfa6
fix: cpubackend
yirongjie Jun 25, 2025
00948f7
feat: add openclbackend
yirongjie Jun 26, 2025
dab4d9d
fix: add openclbackend
yirongjie Jun 26, 2025
de00d80
feat: sage attention && ling-lite-moe
yirongjie Jun 26, 2025
d08cb94
feat: batch infer qwen
yirongjie Jul 15, 2025
aa5b8ca
feat: add opencl qwen2.5-1.5b-q4_0
yirongjie Jul 27, 2025
e92b996
feat: add smallthinker
yirongjie Aug 6, 2025
df0f0b4
feat: add qwen2.5vl
yirongjie Aug 25, 2025
b30be26
fix: vison token purning && qwen2-vl's config
yirongjie Sep 14, 2025
2c46281
fix: qwen3 kai_quant
yirongjie Oct 11, 2025
bb1f5c2
fix: llama3.2-kai
yirongjie Oct 11, 2025
2a11989
feat: add Qwen2-VL QNN
yirongjie Oct 27, 2025
bd6cbfd
rename: src&include -> mllm
yirongjie Oct 27, 2025
82b1159
Merge branch 'main' into main
yirongjie Oct 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .clang-tidy.ignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
src/quantizer/gguf.hpp
mllm/quantizer/gguf.hpp
20 changes: 13 additions & 7 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
.vscode/
.idea/
.cache/
.DS_Store
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove duplicate .DS_Store entry.

Lines 5 and 40 both define .DS_Store. Keep one and remove the duplicate for clarity.

Apply this diff:

- .DS_Store
  examples/test.cpp

Remove line 40 and consolidate at line 5 (already present).

Also applies to: 40-40

🤖 Prompt for AI Agents 
In .gitignore around lines 5 to 40, there is a duplicate .DS_Store entry
(present at line 5 and again at line 40); remove the duplicate at line 40 and
leave the single .DS_Store entry at line 5 so the file contains only one
.DS_Store line for clarity.

build*/
build/
bin/
Expand All @@ -26,15 +27,20 @@ models/*
/.devcontainer/
/.vscode/
workflow.py
src/backends/qnn/qualcomm_ai_engine_direct_220/*
src/backends/qnn/HexagonSDK/*
mllm/backends/qnn/qualcomm_ai_engine_direct_220/*
mllm/backends/qnn/HexagonSDK/*
tmp/
py-build-out/
mllm.egg-info/

examples/demo_deepseek.cpp
src/models/deepseek/*
examples/demo.cpp
mllm/backends/qnn/sdk*

src/backends/qnn/sdk/*
*.mllm


.DS_Store
examples/test.cpp
examples/demo_bailing_moe2*
mllm/models/ling2
scripts/tmp.sh
tools/convertor/gptq_converter.py
*.patch
7 changes: 5 additions & 2 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@
[submodule "third_party/pybind11"]
path = third_party/pybind11
url = https://github.com/pybind/pybind11.git
[submodule "src/backends/xnnpack/third_party/XNNPACK"]
path = src/backends/xnnpack/third_party/XNNPACK
[submodule "mllm/backends/xnnpack/third_party/XNNPACK"]
path = mllm/backends/xnnpack/third_party/XNNPACK
url = https://github.com/google/XNNPACK.git
[submodule "third_party/googletest"]
path = third_party/googletest
url = https://github.com/google/googletest.git
[submodule "mllm/backends/cpu/third_party/kleidiai"]
path = mllm/backends/cpu/third_party/kleidiai
url = https://github.com/ARM-software/kleidiai
Loading