Summary
The encoder currently sets frame_content_size: None in the frame header (frame_compressor.rs:145). The C reference defaults to writing content size when known — this enables decoders to pre-allocate output buffers, significantly improving decompression speed.
C reference behavior
ZSTD_c_contentSizeFlag defaults to 1 (enabled)
- When pledged source size is known, it's written in 1/2/4/8 bytes depending on value
- Single-segment frames (content fits in window) use content size instead of window descriptor — saves 1 byte per frame
Current Rust state
frame_header.rs:74 — FCS field always None
- Header always writes window descriptor even when content size would suffice
frame_compressor.rs:144-148 — single_segment: false always
What needs to be implemented
- Content size tracking — count total bytes read during compression
- Two-pass or buffered approach — either buffer all input to know size upfront, or write placeholder and seek back
- Single-segment optimization — when content ≤ window, set
single_segment: true and omit window descriptor
- FCS encoding — 1/2/4/8 byte encoding based on value
Acceptance criteria
Time estimate
1d
Summary
The encoder currently sets
frame_content_size: Nonein the frame header (frame_compressor.rs:145). The C reference defaults to writing content size when known — this enables decoders to pre-allocate output buffers, significantly improving decompression speed.C reference behavior
ZSTD_c_contentSizeFlagdefaults to 1 (enabled)Current Rust state
frame_header.rs:74— FCS field alwaysNoneframe_compressor.rs:144-148—single_segment: falsealwaysWhat needs to be implemented
single_segment: trueand omit window descriptorAcceptance criteria
Time estimate
1d