Skip to content

On Linux, OGA NPU Execution Mode can't work because of no onnx_custom_ops.so #265

@wadewwwang

Description

@wadewwwang

Following the steps in https://github.com/amd/RyzenAI-SW/blob/main/example/llm/RAG-OGA/README.md to run "python rag.py --direct_llm", it always fails with the below error:
Error: Error encountered while parsing '~/.cache/huggingface/hub/models--amd--Llama-3.2-1B-Instruct-awq-g128-int4-asym-fp16-onnx-hybrid/snapshots/0bad422ccc41500719ba7904200fbd000b56332e/genai_config.json' JSON Error: model:decoder:session_options: Unknown value "custom_allocator" at line 9 index 53

It should be caused by no available onnx_custom_ops.so. So need confirm if OGA NPU Execution Mode can work on Linux and where and when can get this kind of onnx_custom_ops.so for linux

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions