[RUNTIME] Introduce MetadataModule to separate code compilation/interpretation and weight initialization

[zhiics]

This PR attempts to separate metadata initialization and source code compilation (when necessary). The discussion can be found here: https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/15

The goal is to have a standalone wrapper (MetadataModule) to take care of module initialization for different runtimes. In the BYOC case, we save constants and code separately. The constant would be loaded by the wrapper while the code is handled by csourcemodule.

[zhiics]

cc @tqchen @comaniac @junrushao1994 @FrozenGene @lhutton1 @trevor-m @masahi

[zhiics]

@tqchen I removed all wrappers, made MetadataModule internal, and dispatched the initialization of each submodule per function/symbol base. There is now no user API change. But I had to add a field the CSourceModule to let it know what metadata it needs. This, however, doesn't affect the compilation and the original use/semantic of CSourceModule because we only need this information to help build up the MetadataModule.

[tqchen]

cc @mbaret @FrozenGene @comaniac it would be great if you can help to take a look as well

[FrozenGene]

Maybe I miss something. so maybe this is considered. That is I don't see unwrap_modules. I ask this because I think it is related with our next module based interface runtime. The usage like this:

with relay.build_config(opt_level=3):
    complied_graph_lib = relay.build_module.build(
        mod, "llvm", params=params)

from tvm.contrib import util
temp = util.tempdir()
file_name = "deploy_lib.so"
path_lib = temp.relpath(file_name)
complied_graph_lib.export_library(path_lib)
loaded_lib = tvm.runtime.load_module(path_lib)
ctx = tvm.cpu(0)
gmod = loaded_lib['default'](ctx)
set_input = gmod["set_input"]
run = gmod["run"]
get_output = gmod["get_output"]
data = np.random.uniform(-1, 1, size=(1, 3, 224, 224)).astype("float32")
set_input("data", tvm.nd.array(data))
run()
out = get_output(0).asnumpy()

As you could see we will wrap params / lib / json into one lib. So I want to see the logic of unwrap_modules and to see how to connect with it.

THANKS!

[zhiics]

@FrozenGene The current implementation doesn't actually change any user/frontend APIs. We added the MetadataModule to take care of weight initialization and separate it from code compilation. So you would have:

MetadataModule:
    - DSO module
    - Accelerator CSourceModule0
    - Accelerator CSourceModule1

As you can see, MetadataModule manages the params, but we will still have code in the submodules. The serialization and deserialization make sure that the module structure is always maintained correctly. Therefore, we can still have params/lib/json in one module.

In other words, it should not affect the way we package the final artifacts, but it changes the way we compile certain modules (e.g. CSourceModule) and the way we initialize modules that need constant data.

[comaniac]

Overall LGTM. Only miner comments. Thanks!

[comaniac]

LGTM