[GLUTEN-9344][VL] Document dynamic offheap sizing feature#9391
Merged
zhouyuan merged 5 commits intoapache:mainfrom Aug 23, 2025
Merged
[GLUTEN-9344][VL] Document dynamic offheap sizing feature#9391zhouyuan merged 5 commits intoapache:mainfrom
zhouyuan merged 5 commits intoapache:mainfrom
Conversation
Signed-off-by: Yuan Zhou <yuan.zhou@ibm.com>
|
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format? See also: |
zhouyuan
changed the title
srinivasst
reviewed
May 9, 2025
| - Spark first attempts to allocate memory based on the on-heap size. Note that the maximum memory size is controlled by spark.executor.memory. | ||
| - When Velox tries to allocate memory, Gluten attempts to allocate from system memory and records this in the memory allocator. | ||
| - If there is sufficient memory, allocations proceed normally. | ||
| - If memory is insufficient, Spark performs garbage collection (GC) to free on-heap memory, allowing Velox to allocate memory. |
Contributor
There was a problem hiding this comment.
In gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java the borrow code only checks for uncommitted memory (size + usedOffHeapMemory + totalHeapMemory <= TOTAL_MEMORY_SHARED).
It does not acquire memory from the TaskMemoryManager
How will this trigger spill / GC?
Contributor
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
philo-he
approved these changes
Aug 16, 2025
| @@ -0,0 +1,24 @@ | |||
| ## Dynamic Off-heap Sizing | |||
| Spark requires setting both on-heap and off-heap memory sizes, which initializes different memory layouts. Improper configuration of these settings can lead to lower performance. Dynamic off-heap sizing is an experimental feature designed to simplify this process. When enabled, off-heap settings are ignored, and Velox uses the on-heap size as the memory size. | |||
Member
There was a problem hiding this comment.
Nit: recommend to break this into multiple code lines, which is more readable for editors not wrapping text visually. And it can also improve the readability in diffs.
Ditto for other changes.
|
|
||
| ## Limitations | ||
|
|
||
| This feature is still under heavy development. No newline at end of file |
Member
There was a problem hiding this comment.
Suggestion: ** is still in early development.
steveburnett
left a comment
steveburnett
left a comment
There was a problem hiding this comment.
Just a few suggestions of formatting and phrasing.
| ## Dynamic Off-heap Sizing | ||
| Gluten requires setting both on-heap and off-heap memory sizes, which initializes different memory layouts. Improper configuration of these settings can lead to lower performance. | ||
|
|
||
| To fix this issue, Dynamic off-heap sizing is an experimental feature designed to simplify this process. When enabled, off-heap settings are ignored, and Velox uses the on-heap size as the memory size. |
There was a problem hiding this comment.
Suggested change
| To fix this issue, Dynamic off-heap sizing is an experimental feature designed to simplify this process. When enabled, off-heap settings are ignored, and Velox uses the on-heap size as the memory size. | |
| To fix this issue, dynamic off-heap sizing is an experimental feature designed to simplify this process. When enabled, off-heap settings are ignored, and Velox uses the on-heap size as the memory size. |
|
|
||
| In general, the feature works as follows: | ||
|
|
||
| - Spark first attempts to allocate memory based on the on-heap size. Note that the maximum memory size is controlled by spark.executor.memory. |
There was a problem hiding this comment.
Suggested change
| - Spark first attempts to allocate memory based on the on-heap size. Note that the maximum memory size is controlled by spark.executor.memory. | |
| - Spark first attempts to allocate memory based on the on-heap size. Note that the maximum memory size is controlled by `spark.executor.memory`. |
| - If memory is insufficient, Spark performs garbage collection (GC) to free on-heap memory, allowing Velox to allocate memory. | ||
| - If memory remains insufficient after GC, Spark reports an out-of-memory (OOM) issue. | ||
|
|
||
| We then enforce a total memory quota, calculated as the sum of committed and in-use memory in the Java heap (using Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) plus tracked off-heap memory in TreeMemoryConsumer. If an allocation exceeds this total committed memory, the allocation fails, triggering an OOM. |
There was a problem hiding this comment.
Suggested change
| We then enforce a total memory quota, calculated as the sum of committed and in-use memory in the Java heap (using Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) plus tracked off-heap memory in TreeMemoryConsumer. If an allocation exceeds this total committed memory, the allocation fails, triggering an OOM. | |
| We then enforce a total memory quota, calculated as the sum of committed and in-use memory in the Java heap (using `Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()`) plus tracked off-heap memory in `TreeMemoryConsumer`. If an allocation exceeds this total committed memory, the allocation fails and triggers an OOM. |
|
|
||
| We then enforce a total memory quota, calculated as the sum of committed and in-use memory in the Java heap (using Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) plus tracked off-heap memory in TreeMemoryConsumer. If an allocation exceeds this total committed memory, the allocation fails, triggering an OOM. | ||
|
|
||
| With this change, the "quota check" is performed when an allocation in the native engine is informed to Gluten. In practice, this means the Java codebase can oversubscribe memory within the on-heap quota, even if off-heap usage is sufficient to fail the allocation. |
There was a problem hiding this comment.
"is informed to " - can you find another way to describe this? I'm not sure from this phrasing what happens here.
Maybe:
"is sent to Gluten"
"is performed when Gluten receives an allocation request"
or something else you can think of.
zhli1142015
reviewed
Aug 21, 2025
| - If memory is insufficient, Spark performs garbage collection (GC) to free on-heap memory, allowing Velox to allocate memory. | ||
| - If memory remains insufficient after GC, Spark reports an out-of-memory (OOM) issue. | ||
|
|
||
| We then enforce a total memory quota, calculated as the sum of committed and in-use memory in the Java heap (using Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) plus tracked off-heap memory in TreeMemoryConsumer. If an allocation exceeds this total committed memory, the allocation fails, triggering an OOM. |
Contributor
There was a problem hiding this comment.
I think we use only Runtime.getRuntime().totalMemory() for the calculation here.
philo-he
changed the title
Signed-off-by: Yuan <yuanzhou@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
this patch adds documentation for dynamic off-heap sizing feature
fixes: #9344
How was this patch tested?
no need to test