Add onnxruntime as wasi-nn backend

[dongsheng28849455]

Steps to verify:

• Install the onnx runtime (official release), assuming in /opt/onnxruntime
• Build iwasm with WAMR_BUILD_WASI_NN_ONNX enabled
• Using an onnx model of ssd-mobilenetv1 from ONNX Model Zoo
• Generate the data file of input_tensor.bin, from origin picture for wasi-nn (with shape [1, 383, 640, 3])
• Use nn-cli for test, eg.

--load-graph=file=./ssd_mobilenet_v1.onnx,id=graph1,encoding=1 
--init-execution-context=graph-id=graph1,id=exec0 
--set-input=file=./input_tensor.bin,context-id=exec0,dim=1,dim=383,dim=640,dim=3,type=3 
--compute=context-id=exec0 
--get-output=context-id=exec0,file=output.bin

Generate output.bin, with shape [1, 100, 4] and f32 type, which contents match the sample's output

[yamt]

what's the relationship with #4304?

[dongsheng28849455]

what's the relationship with #4304?

1, Adapt to latest wasi-nn arch and support WAMR_BUILD_WASI_EPHEMERAL_NN
2, Test with models and nn-cli

[yamt]

restarting SGX 143
https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/16643007953/job/47097105368?pr=4485

[yamt]

* Using an onnx model of [ssd-mobilenetv1](https://github.com/onnx/models/tree/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1) from [ONNX Model Zoo](https://github.com/onnx/models/blob/main/README.md#onnx-model-zoo)

* Generate the data file of input_tensor.bin, from origin picture for wasi-nn (with shape [1, 383, 640, 3])

* Use [nn-cli](https://github.com/bytecodealliance/wasm-micro-runtime/pull/4373) for test, eg.

--load-graph=file=./ssd_mobilenet_v1.onnx,id=graph1,encoding=1 
--init-execution-context=graph-id=graph1,id=exec0 
--set-input=file=./input_tensor.bin,context-id=exec0,dim=1,dim=383,dim=640,dim=3,type=3 
--compute=context-id=exec0 
--get-output=context-id=exec0,file=output.bin

Generate output.bin, with shape [1, 100, 4] and f32 type, which contents match the sample's output

using this model, i had to use non-zero index for get_output. thus i had to fix nn-cli bug.
which model have you used?

[dongsheng28849455]

* Using an onnx model of [ssd-mobilenetv1](https://github.com/onnx/models/tree/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1) from [ONNX Model Zoo](https://github.com/onnx/models/blob/main/README.md#onnx-model-zoo)

* Generate the data file of input_tensor.bin, from origin picture for wasi-nn (with shape [1, 383, 640, 3])

* Use [nn-cli](https://github.com/bytecodealliance/wasm-micro-runtime/pull/4373) for test, eg.

--load-graph=file=./ssd_mobilenet_v1.onnx,id=graph1,encoding=1 
--init-execution-context=graph-id=graph1,id=exec0 
--set-input=file=./input_tensor.bin,context-id=exec0,dim=1,dim=383,dim=640,dim=3,type=3 
--compute=context-id=exec0 
--get-output=context-id=exec0,file=output.bin

Generate output.bin, with shape [1, 100, 4] and f32 type, which contents match the sample's output

using this model, i had to use non-zero index for get_output. thus i had to fix nn-cli bug. which model have you used?

I'm using this one: https://github.com/onnx/models/blob/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1/model/ssd_mobilenet_v1_10.onnx

[yamt]

I'm using this one: https://github.com/onnx/models/blob/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1/model/ssd_mobilenet_v1_10.onnx

thank you. however, this model looks same in the regard. (have 4 outputs)

[yamt]

I'm using this one: https://github.com/onnx/models/blob/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1/model/ssd_mobilenet_v1_10.onnx

thank you. however, this model looks same in the regard. (have 4 outputs)

maybe you somehow interpreted only the first (idx=0) output, which contains bounding boxes?

[dongsheng28849455]

I'm using this one: https://github.com/onnx/models/blob/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1/model/ssd_mobilenet_v1_10.onnx

thank you. however, this model looks same in the regard. (have 4 outputs)

maybe you somehow interpreted only the first (idx=0) output, which contains bounding boxes?

output shape is [1, 100, 4]:
hexdump output.bin:
0000000 c05e 3ef8 5d0f 3e2f 06ad 3f09 693f 3e5a
0000010 a9d3 3f03 52ae 3d37 4e7d 3f18 1dd4 3e02
0000020 5914 3dc3 41a7 3efa 2644 3f3a 0000 3f80
0000030 73c5 3ef8 af0e 3e65 02ea 3f0b 69e3 3e84
0000040 50f7 3ef7 e514 3e04 d723 3f04 bb62 3e1f
0000050 5a2c 3ef3 75e2 3e84 4f2e 3f0e 329c 3e9e
0000060 c04a 3ef2 ca20 3e99 df0d 3f10 8744 3eb9
0000070 f5c0 3ec3 61ce 3e0f 066a 3ed6 aada 3e1c
0000080 0000 0000 0000 0000 0000 0000 0000 0000

in which, one bounding box for example:
c05e 3ef8 (0.48584265) stands for the ymin
5d0f 3e2f (0.17125343) for xmin
06ad 3f09 (0.5352581) is ymax
693f 3e5a(0.2132921) is xmax

it looks good for my test picure (http://images.cocodataset.org/val2017/000000088462.jpg)

[yamt]

I'm using this one: https://github.com/onnx/models/blob/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1/model/ssd_mobilenet_v1_10.onnx

thank you. however, this model looks same in the regard. (have 4 outputs)

maybe you somehow interpreted only the first (idx=0) output, which contains bounding boxes?

output shape is [1, 100, 4]: hexdump output.bin: 0000000 c05e 3ef8 5d0f 3e2f 06ad 3f09 693f 3e5a 0000010 a9d3 3f03 52ae 3d37 4e7d 3f18 1dd4 3e02 0000020 5914 3dc3 41a7 3efa 2644 3f3a 0000 3f80 0000030 73c5 3ef8 af0e 3e65 02ea 3f0b 69e3 3e84 0000040 50f7 3ef7 e514 3e04 d723 3f04 bb62 3e1f 0000050 5a2c 3ef3 75e2 3e84 4f2e 3f0e 329c 3e9e 0000060 c04a 3ef2 ca20 3e99 df0d 3f10 8744 3eb9 0000070 f5c0 3ec3 61ce 3e0f 066a 3ed6 aada 3e1c 0000080 0000 0000 0000 0000 0000 0000 0000 0000

in which, one bounding box for example: c05e 3ef8 (0.48584265) stands for the ymin 5d0f 3e2f (0.17125343) for xmin 06ad 3f09 (0.5352581) is ymax 693f 3e5a(0.2132921) is xmax

it looks good for my test picure (http://images.cocodataset.org/val2017/000000088462.jpg)

ok. i understood.

actually, the model has 4 outputs as documented
and (with this nn-cli fix) you can get them as the following.

--load-graph=file=ssd_mobilenet_v1_10.onnx,encoding=1 \
--init-execution-context \
--set-input=file=input.bin,dim=1,dim=383,dim=640,dim=3,type=3 \
--compute \
--get-output=idx=0,file=output0.bin \
--get-output=idx=1,file=output1.bin \
--get-output=idx=2,file=output2.bin \
--get-output=idx=3,file=output3.bin

as wasi-nn doesn't have get-output-by-name, you need to use integer indexes.
output tensors of this model are (detection_boxes, detection_classes, detection_scores, num_detections). (in this order)

you were only looking at detection_boxes.

[dongsheng28849455]

I'm using this one: https://github.com/onnx/models/blob/main/validated/vision/object_detection_segmentation/ssd-mobilenetv1/model/ssd_mobilenet_v1_10.onnx

thank you. however, this model looks same in the regard. (have 4 outputs)

maybe you somehow interpreted only the first (idx=0) output, which contains bounding boxes?

output shape is [1, 100, 4]: hexdump output.bin: 0000000 c05e 3ef8 5d0f 3e2f 06ad 3f09 693f 3e5a 0000010 a9d3 3f03 52ae 3d37 4e7d 3f18 1dd4 3e02 0000020 5914 3dc3 41a7 3efa 2644 3f3a 0000 3f80 0000030 73c5 3ef8 af0e 3e65 02ea 3f0b 69e3 3e84 0000040 50f7 3ef7 e514 3e04 d723 3f04 bb62 3e1f 0000050 5a2c 3ef3 75e2 3e84 4f2e 3f0e 329c 3e9e 0000060 c04a 3ef2 ca20 3e99 df0d 3f10 8744 3eb9 0000070 f5c0 3ec3 61ce 3e0f 066a 3ed6 aada 3e1c 0000080 0000 0000 0000 0000 0000 0000 0000 0000
in which, one bounding box for example: c05e 3ef8 (0.48584265) stands for the ymin 5d0f 3e2f (0.17125343) for xmin 06ad 3f09 (0.5352581) is ymax 693f 3e5a(0.2132921) is xmax
it looks good for my test picure (http://images.cocodataset.org/val2017/000000088462.jpg)

ok. i understood.

actually, the model has 4 outputs as documented and (with this nn-cli fix) you can get them as the following.

--load-graph=file=ssd_mobilenet_v1_10.onnx,encoding=1 \
--init-execution-context \
--set-input=file=input.bin,dim=1,dim=383,dim=640,dim=3,type=3 \
--compute \
--get-output=idx=0,file=output0.bin \
--get-output=idx=1,file=output1.bin \
--get-output=idx=2,file=output2.bin \
--get-output=idx=3,file=output3.bin

as wasi-nn doesn't have get-output-by-name, you need to use integer indexes. output tensors of this model are (detection_boxes, detection_classes, detection_scores, num_detections). (in this order)

you were only looking at detection_boxes.

OK, Thank you!

[yamt]

lgtm

[dongsheng28849455]

@lum1n0us could you help review the code?

[yamt]

restarting SGX 143
https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/16904248231/job/47890298413?pr=4485

[lum1n0us]

The code looks good to me. But, could you provide some documentation to guide people on how to use these backends(include a few words about installation)? Most importantly, a smoke test to verify its functionality and serve as a demo would be helpful.

[dongsheng28849455]

The code looks good to me. But, could you provide some documentation to guide people on how to use these backends(include a few words about installation)? Most importantly, a smoke test to verify its functionality and serve as a demo would be helpful.

I wrote some text about how to set up the backend and test it, it may help to understand how to use it.
I can understand it may not enough, But can we merge it first and then add it? Considering the progress of our project

[lum1n0us]

Sure.

[kylo5aby]

LGTM