Skip to content

[Tracking Issue] Support RPC-based execution with AoT Executor #10076

@areusch

Description

@areusch

Right now it's not possible to use the AoT executor on micro over the RPC channel. This issue tracks support for that.

There are a couple pieces here:

  1. Exporting the shape and dtype of parameters so that it can be consumed either from metadata.json or by the runtime.
  2. Writing PackedFunc wrappers around the AoT Executor, similar to Add Module-Based Model Runtime Interface for AOT (support C++ runtime) #9697 (but for C runtime).

Metadata

Metadata

Assignees

No one assigned

    Labels

    core:rpcexecutor:aotruntime components of AOT flow. src/runtime/aot_executor, src/runtime/crt/aot_executor

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions