Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Aug 2, 2023

Which issue does this PR close?

Part of #5885

Rationale for this change

This PR contains new MemoryReservation APIs used in #7130. They were originally written by @tustvold and I polished them up, fixed a bug, and added tests

I broke it into its own PR for ease in reviewing. You can see the usecase in #7130.

What changes are included in this PR?

  1. Add new APIs and tests

Are these changes tested?

Yes

Are there any user-facing changes?

@alamb alamb marked this pull request as draft August 2, 2023 18:17
@alamb alamb force-pushed the alamb/memory_pool_enhancements branch from 6519c23 to de90257 Compare August 2, 2023 18:17
if consumer.can_spill {
self.state.lock().num_spill -= 1;
let mut state = self.state.lock();
state.num_spill = state.num_spill.checked_sub(1).unwrap();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will now panic rather than silently underflow

r1.grow(2000);
assert_eq!(pool.reserved(), 2000);

let mut r2 = MemoryConsumer::new("s1")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed the names to match the local variable names as the indirection of a different number was very confusing to me when debugging this test.

/// the underlying pool.
#[derive(Debug)]
pub struct MemoryReservation {
struct SharedRegistration {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This structure allows different MemoryReservations to share the same consumer

/// # Panics
///
/// Panics if `capacity` exceeds [`Self::size`]
pub fn split(&mut self, capacity: usize) -> MemoryReservation {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here are the new APIs

impl Drop for MemoryReservation {
fn drop(&mut self) {
self.free();
self.policy.unregister(&self.consumer);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is moved to SharedReservation::drop

@alamb alamb marked this pull request as ready for review August 2, 2023 18:20
@alamb alamb merged commit 48766c9 into apache:main Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants