Skip to content

[wasi-nn]Add a new target for llama.cpp as a wasi-nn backend#3709

Merged
wenyongh merged 2 commits intobytecodealliance:mainfrom
lum1n0us:feat/wasi_nn_llamacpp
Sep 10, 2024
Merged

[wasi-nn]Add a new target for llama.cpp as a wasi-nn backend#3709
wenyongh merged 2 commits intobytecodealliance:mainfrom
lum1n0us:feat/wasi_nn_llamacpp

Conversation

@lum1n0us
Copy link
Contributor

@lum1n0us lum1n0us commented Aug 14, 2024 •
edited
Loading

Minimum support:

  • accept (WasmEdge) customized model parameters. metadata.
  • Target wasmedge-ggml examples
    • basic
    • chatml
    • gemma
    • llama
    • qwen

In future. To support if required.

  • Target wasmedge-ggml examples
    • command-r. (>70G memory requirement)
    • embedding. (embedding mode)
    • grammar. (use the grammar option to constrain the model to generate the JSON output)
    • llama-stream. (new APIS compute_single, get_output_single, fini_single)
    • llava. (image representation)
    • llava-base64-stream. (image representation)
    • multimodel. (image representation)
  • Target llamaedge

@lum1n0us lum1n0us changed the title Add a new target for llama.cpp as a wasi-nn backend into cmake framework [wasi-nn]Add a new target for llama.cpp as a wasi-nn backend into cmake framework Aug 14, 2024
@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch 3 times, most recently from fd1cd38 to 2976066 Compare August 16, 2024 02:03
@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch from 2976066 to 4dc980f Compare August 22, 2024 08:46
@lum1n0us lum1n0us changed the title [wasi-nn]Add a new target for llama.cpp as a wasi-nn backend into cmake framework [wasi-nn]Add a new target for llama.cpp as a wasi-nn backend Aug 22, 2024
@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch 2 times, most recently from 7b74f29 to 96d4f87 Compare August 23, 2024 07:16
@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch 3 times, most recently from b30a5b5 to 860cb5d Compare September 4, 2024 04:17
@lum1n0us
Copy link
Contributor Author

lum1n0us commented Sep 5, 2024 •
edited
Loading

core/iwasm/libraries/wasi-nn/test/Dockerfile.wasi-nn-smoke needs more work. Will update it in another PR.

@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch from 860cb5d to f893c7f Compare September 6, 2024 00:25
@lum1n0us lum1n0us marked this pull request as ready for review September 6, 2024 00:25
@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch 4 times, most recently from 22174e2 to 2dcb65b Compare September 7, 2024 06:52
wenyongh
wenyongh reviewed Sep 9, 2024
View reviewed changes
@lum1n0us lum1n0us force-pushed the feat/wasi_nn_llamacpp branch from 2dcb65b to 10965ba Compare September 9, 2024 10:02
@wenyongh wenyongh merged commit 0599351 into bytecodealliance:main Sep 10, 2024
@lum1n0us lum1n0us deleted the feat/wasi_nn_llamacpp branch November 20, 2024 05:58
yamt added a commit to yamt/wasm-micro-runtime that referenced this pull request Jul 14, 2025
@yamt
wasi-nn: add a comment on load_by_name_with_config
12cd7d4 
this api has been introduced by
bytecodealliance#3709
in which i couldn't find any explanations.

this comment is my best guess.
@yamt yamt mentioned this pull request Jul 14, 2025
Merged
yamt added a commit to yamt/wasm-micro-runtime that referenced this pull request Jul 15, 2025
@yamt
wasi-nn: add a comment on load_by_name_with_config
6581e8c 
this api has been introduced by
bytecodealliance#3709
in which i couldn't find any explanations.

this comment is my best guess.
lum1n0us pushed a commit that referenced this pull request Jul 17, 2025
@yamt
wasi-nn: add a comment on load_by_name_with_config (#4492)
e4d2673 
this api has been introduced by
#3709
in which i couldn't find any explanations.

this comment is my best guess.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@wenyongh wenyongh wenyongh approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lum1n0us @wenyongh