Skip to content
Merged
3 changes: 2 additions & 1 deletion BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,9 @@ encoder:

- `structured-zstd::Fastest` vs `zstd` level `1`
- `structured-zstd::Default` vs `zstd` level `3`
- `structured-zstd::Better` vs `zstd` level `7`

`Better` and `Best` are intentionally excluded until the encoder implements them.
`Best` is intentionally excluded until the encoder implements it.

Dictionary benchmarks are tracked separately with C FFI `with_dict` vs `without_dict` runs, using a
dictionary trained from scenario samples. Pure Rust dictionary compression is still pending and is
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This is a **maintained fork** of [KillingSpark/zstd-rs](https://github.com/Killi
**Fork goals:**
- Dictionary compression improvements (critical for per-label trained dictionaries in LSM-tree)
- Performance parity with C zstd for decompression (currently 1.4-3.5x slower)
- Additional compression levels (Default/Better/Best — currently only Fastest is implemented)
- Additional compression levels (Best still pending — Fastest, Default, and Better are implemented)
- No FFI — pure `cargo build`, no cmake/system libraries (ADR-013 compliance)

**Upstream relationship:** We periodically sync with upstream but maintain an independent development trajectory focused on CoordiNode requirements.
Expand All @@ -44,7 +44,7 @@ Complete RFC 8878 implementation. Performance: ~1.4-3.5x slower than C zstd depe
- [x] Uncompressed blocks
- [x] Fastest (roughly level 1)
- [x] Default (roughly level 3)
- [ ] Better (roughly level 7)
- [x] Better (roughly level 7)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- [ ] Best (roughly level 11)
- [x] Checksums
- [x] Dictionary compression
Expand Down
8 changes: 5 additions & 3 deletions cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,15 @@ enum Commands {
/// - 0: Uncompressed
/// - 1: Fastest
/// - 2: Default
/// - 3: Better (lazy2, ~zstd level 7)
///
/// Streaming mode currently supports only levels 0..=2.
/// Streaming mode currently supports only levels 0..=3.
#[arg(
short,
long,
value_name = "COMPRESSION_LEVEL",
default_value_t = 2,
value_parser = clap::value_parser!(u8).range(0..=2),
value_parser = clap::value_parser!(u8).range(0..=3),
verbatim_doc_comment
)]
level: u8,
Expand Down Expand Up @@ -107,6 +108,7 @@ fn compress(input: PathBuf, output: PathBuf, level: u8) -> color_eyre::Result<()
0 => CompressionLevel::Uncompressed,
1 => CompressionLevel::Fastest,
2 => CompressionLevel::Default,
3 => CompressionLevel::Better,
_ => return Err(eyre!("unsupported compression level: {level}")),
};
ensure_distinct_paths(&input, &output)?;
Expand Down Expand Up @@ -400,7 +402,7 @@ mod tests {

#[test]
fn cli_rejects_unsupported_compression_level_at_parse_time() {
let parse = Cli::try_parse_from(["structured-zstd", "compress", "in.bin", "--level", "3"]);
let parse = Cli::try_parse_from(["structured-zstd", "compress", "in.bin", "--level", "4"]);
assert!(parse.is_err());
}

Expand Down
7 changes: 6 additions & 1 deletion zstd/benches/support/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ pub(crate) fn benchmark_scenarios() -> Vec<Scenario> {
scenarios
}

pub(crate) fn supported_levels() -> [LevelConfig; 2] {
pub(crate) fn supported_levels() -> [LevelConfig; 3] {
[
LevelConfig {
name: "fastest",
Expand All @@ -82,6 +82,11 @@ pub(crate) fn supported_levels() -> [LevelConfig; 2] {
rust_level: CompressionLevel::Default,
ffi_level: 3,
},
LevelConfig {
name: "better",
rust_level: CompressionLevel::Better,
ffi_level: 7,
},
]
}

Expand Down
10 changes: 6 additions & 4 deletions zstd/src/encoding/frame_compressor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -277,10 +277,12 @@ impl<R: Read, W: Write, M: Matcher> FrameCompressor<R, W, M> {
header.serialize(output);
output.extend_from_slice(&uncompressed_data);
}
CompressionLevel::Fastest | CompressionLevel::Default => {
// Default shares this fast block-encoding pipeline, but it
// remains a distinct level via the matcher's dfast backend.
compress_fastest(&mut self.state, last_block, uncompressed_data, output)
CompressionLevel::Fastest
| CompressionLevel::Default
| CompressionLevel::Better => {
// All compressed levels share this block-encoding pipeline;
// they differ only in the matcher backend and its parameters.
compress_block_encoded(&mut self.state, last_block, uncompressed_data, output)
}
_ => {
unimplemented!();
Expand Down
8 changes: 6 additions & 2 deletions zstd/src/encoding/levels/fastest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@ use crate::{
};
use alloc::vec::Vec;

/// Compresses a single block at [`crate::encoding::CompressionLevel::Fastest`].
/// Compresses a single block using the shared compressed-block pipeline.
///
/// Used by all compressed levels (Fastest, Default, Better). The actual
/// compression quality is determined by the matcher backend in `state`,
/// not by this function.
///
/// # Parameters
/// - `state`: [`CompressState`] so the compressor can refer to data before
Expand All @@ -17,7 +21,7 @@ use alloc::vec::Vec;
/// larger input
/// - `output`: As `uncompressed_data` is compressed, it's appended to `output`.
#[inline]
pub fn compress_fastest<M: Matcher>(
pub fn compress_block_encoded<M: Matcher>(
state: &mut CompressState<M>,
last_block: bool,
uncompressed_data: Vec<u8>,
Expand Down
2 changes: 1 addition & 1 deletion zstd/src/encoding/levels/mod.rs
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
mod fastest;
pub use fastest::compress_fastest;
pub use fastest::compress_block_encoded;
Loading
Loading