Skip to content

chore: Add allocation source to StreamReader#332

Merged
viirya merged 2 commits intoapache:mainfrom
viirya:add_root_source
Apr 26, 2024
Merged

chore: Add allocation source to StreamReader#332
viirya merged 2 commits intoapache:mainfrom
viirya:add_root_source

Conversation

@viirya
Copy link
Copy Markdown
Member

@viirya viirya commented Apr 26, 2024

Which issue does this PR close?

Closes #.

Rationale for this change

There is a memory leak reported by Java Arrow in #324. When looking at the error message for debugging:

Memory was leaked by query. Memory leaked: (49152)
Allocator(ROOT) 0/49152/180352/9223372036854775807 (res/actual/peak/limit)

I figured that it is hard to know where the allocation comes from. In this PR, I added the source of allocators.

The error message is updated to:

Memory was leaked by query. Memory leaked: (49152)                                                                                                                                                                                     
Allocator(StreamReader/shuffle reader) 0/49152/180352/9223372036854775807 (res/actual/peak/limit)                                                                                                                                      

What changes are included in this PR?

How are these changes tested?

@viirya
Copy link
Copy Markdown
Member Author

viirya commented Apr 26, 2024

cc @andygrove @sunchao @comphead

Copy link
Copy Markdown
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @viirya

@viirya viirya merged commit 897dde7 into apache:main Apr 26, 2024
@viirya
Copy link
Copy Markdown
Member Author

viirya commented Apr 26, 2024

Merged. Thanks.

@viirya viirya deleted the add_root_source branch April 26, 2024 21:55
himadripal pushed a commit to himadripal/datafusion-comet that referenced this pull request Sep 7, 2024
* chore: Add allocation source to StreamReader

* Use simple name
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants