Running single node graphs on NPU

[RachidFK]

Hello,

I am trying to run a single-node graph (Add, Concat) with fixed tensor shapes on NPU, but the compiler keeps assigning the node to execute on the CPU.

I created the single-node ONNX graph and converted/quantized it using Quark to BF16. When running the model, VitisAI EP is always executing the node on the CPU. Based on the documentation, both operators should be supported on the NPU. I used the config.json file from the Whisper demo. Please find my system configuration below.

Can we run single-node graphs (single operators) on NPU? If so, could you please help me figure out how to do it?

Thank you,

NPU: STX
RyzenAI-SW Version: 1.5.1
OS: Windows 11
Driver Version: 32.0.203.329 (12/4/2025)

[dwithchenna]

@RachidFK can you provide some more details on the specific node you are trying to run, here is list of supported operations for reference, Link: https://ryzenai.docs.amd.com/en/latest/ops_support.html

Generally speaking, it might not be most efficient to run a single node, I would recommend consider running a sub-graph of the model at least to make best use of NPU.

I also see that you are using RAI 1.5.1, please try with the latest release of RAI 1.7 and NPU driver version: 32.0.203.280

[RachidFK]

@dwithchenna Of course! Please find an example single-node graph I tried to run.

I agree with you, it might not be the most efficient way to run a single node. However, I am trying to run some this experiments that require running single operators on the NPU.

As for the RAI version, I need to run the experiments with RAI 1.5.1 to ensure compatibility with previous experiments I have ran. I am using these experiments for a research paper, and to avoid having to re-run everything, it would be best if I can complete the single-node experiment with RAI 1.5.1. Once, I complete them, I will make sure to update RAI and the driver.

concat_graphs.zip