Add support for ListView

[grod220]

Motivation

The existing PodSlice had several limitations:

• Limited API: The API lacked common Vec-like methods such as remove.
• Hardcoded Length Type: It always used a PodU32 for the length, offering no flexibility for smaller or larger collections.
• No Alignment Handling: It did not account for alignment requirements of the element type T. If the data array was not naturally aligned after the u32 length prefix, it could lead to alignment faults or misinterpretation of data.

Changes made

This PR introduces a new list module that provides a fully‑featured, zero‑copy, variable‑length collection API: ListView, ListViewMut, & ListViewReadOnly.

Has general‑purpose length prefixes – PodLength lets any of the pod‑ints act as the length type.

PodSlice and PodSliceMut are now marked as deprecated.

[joncinque]

Mostly small points, looks good to me!

As another naming idea, what we're really providing is a View on existing data, so it's like a Vec that won't reallocate, which also similar to an arena allocator.

So we could go with ListView or NoAllocVec or ArenaList. I kind of like the last one, since it might describe the situation best.

[febo]

Looks very nice – left a few comments. The main one is about the alignment. It would be nice if we could make it more flexible to support types 8-byte aligned.

[febo]

So we could go with ListView or NoAllocVec or ArenaList. I kind of like the last one, since it might describe the situation best.

I have a slight preference for ListView – I am not sure if users will make the connection between arena allocator and ArenaList. 😊

[buffalojoec]

So we could go with ListView or NoAllocVec or ArenaList. I kind of like the last one, since it might describe the situation best.

I have a slight preference for ListView – I am not sure if users will make the connection between arena allocator and ArenaList. 😊

Agreed, I personally have never heard the term "arena" in this context before, and I assume most developers will whiff on the underlying meaning at first. However, "view" implies readonly, and that's not the case here.

Here are a few ideas:

• PodList (I think it's actually ok)
• StackList
• BoxList
• Dynamic + any of the above
• PodSpan

[febo]

So we could go with ListView or NoAllocVec or ArenaList. I kind of like the last one, since it might describe the situation best.

I have a slight preference for ListView – I am not sure if users will make the connection between arena allocator and ArenaList. 😊

Agreed, I personally have never heard the term "arena" in this context before, and I assume most developers will whiff on the underlying meaning at first. However, "view" implies readonly, and that's not the case here.

Here are a few ideas:

• PodList (I think it's actually ok)
• StackList
• BoxList
• Dynamic + any of the above
• PodSpan

Agree with your point on the "View" implying read-only, but while the type has a "dynamic" size its capacity is fixed. In that sense it is a view. 😊

The only concern that I have with using Pod* is that the type itself is not Pod, so it might look a bit odd.

[grod220]

Sorry for the delay on this! Wasn't quite sure we were going to need this until recent confirmations in the Unstake program.

[buffalojoec]

Awesome!! This is looking fantastic, and the diagrams under unpack_internal are really an elegant touch for helping follow the slice processing.

[grod220]

A few things I am realizing.

1 - With unstake program’s new settlement design, we no longer need the remove functions.
2 - Users of ListView will not be able to use PodSlice for read-only use (if they rely upon the default len bytes). Pod64 vs Pod32 defaults. Given ListView now has iterable methods, it sort of seems like it could replace both PodSlice & PodSliceMut.

Up for discussion if this is worth keeping or not. The benefits are better alignment handling, remove functions, and test coverage. But perhaps these features are not pressing any longer.

[buffalojoec]

1 - With unstake program’s new settlement design, we no longer need the remove functions. 2 - Users of ListView will not be able to use PodSlice for read-only use (if they rely upon the default len bytes). Pod64 vs Pod32 defaults. Given ListView now has iterable methods, it sort of seems like it could replace both PodSlice & PodSliceMut.

Up for discussion if this is worth keeping or not. The benefits are better alignment handling, remove functions, and test coverage. But perhaps these features are not pressing any longer.

Are we totally sure we don't need it yet, though? I know we may not need remove but what about iter_mut? I could see that being useful. What about just marking this as a draft until we finish the settlement implementation?

[grod220]

I know we may not need remove but what about iter_mut?

Yes, iterables on the mutable version are definitely useful. At the moment, it's awkward. We are reading as PodSlice, doing iteration, and then immediately later reading the bytes as PodSliceMut to make edits. With the fixes for this + alignment updates, I think there is an argument to ship this regardless. However, the more full-featured ListView becomes, it sort of doesn't make sense for PodSlice to be used at all and that should be likewise deprecated.

[buffalojoec]

However, the more full-featured ListView becomes, it sort of doesn't make sense for PodSlice to be used at all and that should be likewise deprecated.

Yep, I would agree!

[joncinque]

Looks great overall! Mostly small things, that you can take or leave in future work

[grod220]

Having the mutable ListView methods sort of made it a necessity to have a read-only view of the same data (given it was not compatible with PodSlice). This prompted a sizable change to the API. Curious to have some thoughts from others on this approach.

New API pattern

ListView - the entrypoint/factory with unpack methods that return:

• ListViewMut - Mutable view operations
• ListViewReadOnly - Immutable view operations

Both of them implement ListViewable which guarantees a set of common methods so we don't have issues like .iter() or len() being on one and not the other.

  let size = ListView::<u32>::size_of(3).unwrap();
  let mut buffer = vec![0u8; size];
  let mut list = ListView::init(&mut buffer).unwrap();

  list.push(42u32).unwrap();
  list.push(100u32).unwrap();

  // Later, unpack the same buffer as read-only
  let read_list = ListView::<u32>::unpack(&buffer).unwrap();
  assert_eq!(read_list.len(), 2);

[joncinque]

Just some small things, looking great overall!

[joncinque]

!

[buffalojoec]

Nice!! This is super useful.