Skip to content

Improve Buffer documentation, deprecate Buffer::from_bytes add From<Bytes> and From<bytes::Bytes> impls#6939

Merged
alamb merged 5 commits intoapache:mainfrom
alamb:alamb/better_bytes_docs
Jan 6, 2025
Merged

Improve Buffer documentation, deprecate Buffer::from_bytes add From<Bytes> and From<bytes::Bytes> impls#6939
alamb merged 5 commits intoapache:mainfrom
alamb:alamb/better_bytes_docs

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Jan 4, 2025

Which issue does this PR close?

Rationale for this change

As described by @tv42 #6754 and @kylebarron in #6930 creating Buffers in a zero copy way is confusing, I have also been personally confused by Bytes and bytes::Bytes difference

Also, the fact it is not clear how to make a Buffer in a zero copy manner lead to unintended copies as described by @XiangpengHao in

At least part of this confusion is that Bytes is an internal (non pub) struct that is named the same as bytes::Bytes but appears in the public API

I would like to make the whole situation less confusing

What changes are included in this PR?

  1. Add documentation to Bytes making it clear it is internal and how it is related to bytes::Bytes
  2. Improve documentation on Buffer and add examples of how to construct Buffer from Vec and bytes::Bytes
  3. Add From impls to make it easier to create Buffer in zero copy manner (used in examples)
  4. Deprecate from_bytes to close Public API using private types: Buffer::from_bytes takes unexported Bytes #6754
  5. Update parquet reader to use new From impls
  6. Misc other documentation improvements

Are there any user-facing changes?

  1. Better Docs
  2. From impls for Buffer

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Public API using private types: Buffer::from_bytes takes unexported Bytes

3 participants