PR #4959 expanded MiniBlockChunk minichunk size from u16 to u32, introducing new metadata and protobuf properties. The documentation at docs/src/format/file/encoding.md should be updated to reflect these changes:
- New protobuf property:
has_large_chunk for MiniBlockLayout.
- New metadata key:
lance-encoding:minichunk-size for specifying mini chunk size (default 4KB, can be larger, e.g., 64KB).
buffer_sizes now stored as u32 in memory:
- Lance format v2.1: still stored as
u16 on disk, converted to u32 in memory.
- Lance format v2.2: default mini chunk size remains 4KB, but users can override it (up to 64KB or larger) via the new metadata key.
The docs should explain these changes so users and developers understand the updated MiniBlockChunk behavior.
PR #4959 expanded
MiniBlockChunkminichunk size fromu16tou32, introducing new metadata and protobuf properties. The documentation atdocs/src/format/file/encoding.mdshould be updated to reflect these changes:has_large_chunkforMiniBlockLayout.lance-encoding:minichunk-sizefor specifying mini chunk size (default 4KB, can be larger, e.g., 64KB).buffer_sizesnow stored asu32in memory:u16on disk, converted tou32in memory.The docs should explain these changes so users and developers understand the updated
MiniBlockChunkbehavior.