Skip to content

[Texture support][Part 3] Support storage scope tag in graph runtime codegen, planning, runtime and compile engine #7688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
csullivan wants to merge 13 commits into from
Closed

[Texture support][Part 3] Support storage scope tag in graph runtime codegen, planning, runtime and compile engine #7688

csullivan wants to merge 13 commits into from

Conversation

@csullivan
Copy link
Contributor

@csullivan csullivan commented Mar 18, 2021 •
edited
Loading

This PR adds generic support for the use of storage scope in Relay's graph runtime lowering path. The key conceptual additions made are:

  • Introduce a target dependent CollectStorageInfo pass which returns a storage mapping for the outputs of each Expr in a Relay function. The consumers of the output storage scope annotations are:
    • The graph runtime memory planner, which can utilize the output storage scope when doing token allocation optimization
    • The graph runtime codegen, which annotates the graph json with these storage scopes.
    • The target dependent CollectBufferBinds packed function which translates these storage scopes into buffers to be bound when lowering a Relay primitive func in CompileEngine (via the binds field of tvm::lower/build)
    • Finally the graph runtime, which utilizes the storage scope annotations in the graph json when setting up data space storage

The two introduced CollectStorageInfo and CollectBufferBinds are dispatched to via a target dependent packed function. If no implementation exists for a given target, the storage scope of each Expr defaults to global scope and the runtime is unaffected.

See RFC here: https://discuss.tvm.apache.org/t/rfc-texture-memory-support/9467

csullivan added 13 commits March 17, 2021 12:55
@csullivan
Add node_storage_scope_ map and field to storage token.
d725d49
@csullivan
Use tuple for output memory plan info.
f3bf4f9
@csullivan
Add storage_scope to memory planner output
3ab43ab 
and serialize/deserialize in graph_runtime.
@csullivan
Add CollectStorageInfo declaration for
976c4e5 
use in graph memory planner.
@csullivan
Temporarily hard code intermediate texture storage ids to be
ec46f95 
persistent and reallocable via pools. This will need to
change but serves as a starting point.
@csullivan
Use storage scope attribute when doing storage allocations in
ed43583 
GraphRuntime::SetupStorage().
@csullivan
Support passing buffer binds to compile engine during lowering.
8011e57
@csullivan
Introduce CollectBufferBinds packed func which
30215b9 
can provide buffers to be bound during lower. The
packed func is specialized over target
(and target attr:device) type.
@csullivan
Properly handle storage scope for multiple output nodes.
130cf5f
@csullivan
Add bound buffers to compile engine cache key.
9bc10af
@csullivan
Change TargetsMap in GRC to from std::unordered_map to tvm::Map.
5aaf9db
@csullivan
Dispatch to target specific storage scope
a9e62e5 
collection function (CollectStorageInfo) in memory planning.
This allows users to implement a target specific storage
mapping as part of memory planning that depends on the
target specific constraints.
@csullivan
Rename target prefix
53cfedb
@csullivan csullivan mentioned this pull request Mar 18, 2021
Closed
@tmoreau89 tmoreau89 mentioned this pull request Jan 10, 2022
Closed
@masahi masahi mentioned this pull request Jan 10, 2022
Closed
@jroesch jroesch added needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it status: need review labels Jan 19, 2022
@csullivan
Copy link
Contributor Author

csullivan commented Oct 14, 2022

Subsumed by #11875.

@csullivan csullivan closed this Oct 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it status: need review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@csullivan @jroesch