[microNPU] Calculate memory pressure for microNPU external functions #11209
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
During the microNPU compilation stage, the "used_memory" annotations on external microNPU functions are read to determine a memory pressure value. This value is passed to the cascader to better approximate the memory available for the optimization. Change-Id: I11a311b0005e785637014cb451f4aed96edcda26
Change-Id: I41acfc83f05b2204075edb99f86a0eecaba00f71
Contributor
Author
|
This is now green :) |
Contributor
|
Thanks @lhutton1! broadly looks good. Just one small thing -- can we add a full offload test case with and without I/O buffer re-use as these are more frequently occurring scenarios ? |
Change-Id: If3e672d402ab237fa82e34761bb972d2e9483ba9
Contributor
Author
|
Thanks for the review @manupa-arm, I added the test cases! |
Contributor
|
Thanks @lhutton1 ! this is merged now! |
blackkker
pushed a commit
to blackkker/tvm
that referenced
this pull request
Jul 7, 2022
…pache#11209) * [microNPU] Calculate memory pressure for microNPU external functions During the microNPU compilation stage, the "used_memory" annotations on external microNPU functions are read to determine a memory pressure value. This value is passed to the cascader to better approximate the memory available for the optimization. Change-Id: I11a311b0005e785637014cb451f4aed96edcda26 * fix get size from memory region Change-Id: I41acfc83f05b2204075edb99f86a0eecaba00f71 * add test case for full offload Change-Id: If3e672d402ab237fa82e34761bb972d2e9483ba9
masahi
pushed a commit
to masahi/tvm
that referenced
this pull request
Jul 15, 2022
…pache#11209) * [microNPU] Calculate memory pressure for microNPU external functions During the microNPU compilation stage, the "used_memory" annotations on external microNPU functions are read to determine a memory pressure value. This value is passed to the cascader to better approximate the memory available for the optimization. Change-Id: I11a311b0005e785637014cb451f4aed96edcda26 * fix get size from memory region Change-Id: I41acfc83f05b2204075edb99f86a0eecaba00f71 * add test case for full offload Change-Id: If3e672d402ab237fa82e34761bb972d2e9483ba9
mikeseven
pushed a commit
to mikeseven/tvm
that referenced
this pull request
Sep 27, 2023
…pache#11209) * [microNPU] Calculate memory pressure for microNPU external functions During the microNPU compilation stage, the "used_memory" annotations on external microNPU functions are read to determine a memory pressure value. This value is passed to the cascader to better approximate the memory available for the optimization. Change-Id: I11a311b0005e785637014cb451f4aed96edcda26 * fix get size from memory region Change-Id: I41acfc83f05b2204075edb99f86a0eecaba00f71 * add test case for full offload Change-Id: If3e672d402ab237fa82e34761bb972d2e9483ba9
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
During the microNPU compilation stage, the "used_memory" annotations on external microNPU functions are read to determine a memory pressure value. This value is passed to the cascader to better approximate the memory available for the optimization.
Note: this PR relies on #11091 and #11208 so this PR also contains the contents of those PR's.cc @Mousius @NicolaLancellotti @ekalda @manupa-arm