Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
fd9e3e5
docs(readme): managed fork header, donate QR, FUNDING.yml
polaz Mar 25, 2026
b555217
chore(crate): rename to structured-zstd v0.0.1, update metadata
polaz Mar 25, 2026
eac7e99
ci: add CI matrix, release-plz, codecov, nextest, coderabbit
polaz Mar 25, 2026
742d249
test(bench): add FFI comparison benchmark, fix crate rename references
polaz Mar 25, 2026
33af678
docs(readme): fix import paths to structured_zstd, absolute QR URL, f…
polaz Mar 25, 2026
1caba3f
ci(bench): github-action-benchmark with 5 decompress/compress variations
polaz Mar 25, 2026
d2faa13
chore(license): fill Apache-2.0 copyright placeholder
polaz Mar 25, 2026
1e825e1
fix(ci): replace --all-features with explicit features, exclude rustc…
polaz Mar 25, 2026
f897b6b
fix(docs): update include_str path after Readme.md → README.md rename
polaz Mar 25, 2026
a72459f
fix(dict): isize → i64 in Karp-Rabin hash for 32-bit targets
polaz Mar 25, 2026
a3ee043
test(integrity): add 5000-iteration roundtrip integrity tests
polaz Mar 25, 2026
28c7bc7
fix(fork): retain upstream MIT notice, fix bench black_box, pipefail
polaz Mar 25, 2026
75ef90c
fix(lint): suppress dead_code warnings on test-only roundtrip helpers
polaz Mar 25, 2026
d4fcdce
fix(docs): replace ruzstd:: with structured_zstd:: in all doc examples
polaz Mar 25, 2026
29ecea3
fix(dict): correct Karp-Rabin h precompute loop and signed modulo
polaz Mar 25, 2026
8c7e646
fix(dict): guard empty pattern in estimate_frequency, pin QR URL to main
polaz Mar 25, 2026
9553c11
fix(ci): add -p structured-zstd for workspace feature resolution, upd…
polaz Mar 25, 2026
be3dc74
fix(ci): set working-directory to ruzstd/ for nextest (corpus file pa…
polaz Mar 25, 2026
4206820
refactor(workspace): rename ruzstd/ → zstd/, purge all ruzstd references
polaz Mar 25, 2026
98677f3
fix(dict): fix occurances → occurrences typo in dictionary module
polaz Mar 25, 2026
c9d941e
fix(cli): update homepage, repository, license to match fork
polaz Mar 25, 2026
c60f020
fix(bench): symmetric per-iteration allocation in decompress benchmark
polaz Mar 25, 2026
412dce8
fix(test): mark corpus files as binary to prevent CRLF conversion on …
polaz Mar 25, 2026
28b0d75
fix(test): use recursive glob in .gitattributes for dict_tests subdirs
polaz Mar 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 5 additions & 0 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

reviews:
auto_apply_labels: true
profile: assertive
10 changes: 10 additions & 0 deletions .config/nextest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[store]
dir = "target/nextest"

[profile.ci]
retries = 2
fail-fast = false
slow-timeout = { period = "120s", terminate-after = 2 }

[profile.ci.junit]
path = "junit.xml"
1 change: 1 addition & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
github: structured-world
Comment thread
polaz marked this conversation as resolved.
15 changes: 15 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# GitHub Copilot Instructions for structured-zstd

## Project Overview

Pure Rust zstd implementation — managed fork of [ruzstd (KillingSpark/zstd-rs)](https://github.com/KillingSpark/zstd-rs). Focus: dictionary compression improvements and performance parity with C zstd for CoordiNode LSM-tree.

## Review Scope Rules

**Review ONLY code within the PR's diff.** For issues found outside the diff, suggest creating a separate issue.

## Rust Code Standards

- **Clippy:** Must pass `cargo clippy --all-features -- -D warnings`
- This is a fork — avoid suggesting architectural changes that diverge too far from upstream
- Performance-critical code: benchmark before/after any changes
22 changes: 22 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
version: 2
updates:
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
groups:
minor-and-patch:
patterns:
- "*"
update-types:
- "minor"
- "patch"
commit-message:
prefix: "chore(deps)"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
commit-message:
prefix: "ci(deps)"
43 changes: 43 additions & 0 deletions .github/scripts/run-benchmarks.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/bin/bash
# Run compare_ffi benchmarks and produce github-action-benchmark JSON.
# Output: benchmark-results.json (customSmallerIsBetter format — lower time = better)
set -eo pipefail

echo "Running benchmarks..." >&2

# Run criterion benchmarks, capture output
cargo bench --bench compare_ffi -p structured-zstd -- --output-format bencher | tee /tmp/bench-raw.txt

echo "Parsing results..." >&2

# Parse criterion bencher output into github-action-benchmark JSON
# Format: "test <name> ... bench: <ns> ns/iter (+/- <variance>)"
python3 - <<'PYEOF'
import json, re, sys

results = []
with open("/tmp/bench-raw.txt") as f:
for line in f:
m = re.match(r"test (\S+)\s+\.\.\. bench:\s+([\d,]+) ns/iter", line)
if m:
name = m.group(1)
ns = int(m.group(2).replace(",", ""))
# Convert ns to ms for readability
ms = ns / 1_000_000
results.append({
"name": name,
"unit": "ms",
"value": round(ms, 3),
})

if not results:
print("ERROR: No benchmark results parsed!", file=sys.stderr)
sys.exit(1)

with open("benchmark-results.json", "w") as f:
json.dump(results, f, indent=2)

print(f"Wrote {len(results)} benchmark results to benchmark-results.json", file=sys.stderr)
for r in results:
print(f" {r['name']}: {r['value']} {r['unit']}", file=sys.stderr)
PYEOF
181 changes: 141 additions & 40 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,48 +1,149 @@
on: [push, pull_request]

name: CI

on:
push:
branches: [main]
pull_request:
branches: [main]

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always

jobs:
tests:
name: stable lint, test, build
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v4

- name: Install stable toolchain
uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- name: Install cargo-hack and cargo-msrv
uses: taiki-e/install-action@v2
with:
tool: cargo-hack, cargo-msrv
- name: check
run: cargo hack check --workspace --feature-powerset --exclude-features rustc-dep-of-std
- name: cargo hack clippy
run: cargo hack clippy --workspace --feature-powerset --exclude-features rustc-dep-of-std
- name: cargo hack test
run: cargo hack test --workspace --feature-powerset --exclude-features rustc-dep-of-std
- name: Verify MSRV (cli)
run: cargo msrv verify --path cli/
- name: Verify MSRV (lib)
run: cargo msrv verify --path ruzstd/

nightly-stuff:
name: nightly lint, miri
lint:
timeout-minutes: 10
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v4
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
- name: Format
run: cargo fmt --all -- --check
- name: Clippy
run: cargo clippy -p structured-zstd --features hash,std,dict_builder -- -D warnings

test:
needs: lint
timeout-minutes: 15
strategy:
fail-fast: true
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
Comment thread
polaz marked this conversation as resolved.
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: taiki-e/install-action@nextest
- uses: Swatinem/rust-cache@v2
with:
prefix-key: ${{ runner.os }}-cargo
- name: Test
working-directory: zstd
run: cargo nextest run --profile ci -p structured-zstd --features hash,std,dict_builder
- name: Doc tests
Comment thread
polaz marked this conversation as resolved.
Comment thread
polaz marked this conversation as resolved.
run: cargo test --doc -p structured-zstd --features hash,std,dict_builder

- name: Install nightly toolchain
uses: dtolnay/rust-toolchain@nightly
cross-i686:
needs: lint
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
targets: i686-unknown-linux-gnu
- uses: taiki-e/install-action@nextest
- name: Install 32-bit libs
run: sudo apt-get update && sudo apt-get install -y gcc-multilib
- uses: Swatinem/rust-cache@v2
with:
components: rustfmt, clippy, miri
prefix-key: cross-i686
- name: Test (i686)
working-directory: zstd
run: cargo nextest run --profile ci -p structured-zstd --features hash,std,dict_builder --target i686-unknown-linux-gnu

- run: cargo +nightly fmt --all -- --check
- run: cargo +nightly clippy --workspace --no-default-features -- -D warnings
- run: cargo +nightly clippy --workspace -- -D warnings
- run: cargo +nightly miri test ringbuffer
- run: cargo +nightly miri test short_Writer
msrv:
needs: lint
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
toolchain: "1.87"
- uses: taiki-e/install-action@nextest
- uses: Swatinem/rust-cache@v2
with:
prefix-key: msrv
- name: Test (MSRV)
working-directory: zstd
run: cargo nextest run --profile ci -p structured-zstd --features hash,std,dict_builder

codecov:
needs: lint
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@nightly
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov -p structured-zstd --features hash,std,dict_builder --lcov --output-path lcov.info
working-directory: zstd
- uses: codecov/codecov-action@v5
with:
files: zstd/lcov.info
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

benchmark:
name: Performance regression check
needs: lint
timeout-minutes: 30
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6

- name: Generate bot token
id: bot-token
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
uses: actions/create-github-app-token@v3
with:
app-id: ${{ secrets.RELEASER_APP_ID }}
private-key: ${{ secrets.RELEASER_APP_PRIVATE_KEY }}

- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
prefix-key: bench

- name: Run benchmarks
run: bash .github/scripts/run-benchmarks.sh

- name: Store benchmark results
if: steps.bot-token.outputs.token != ''
uses: benchmark-action/github-action-benchmark@v1
with:
name: "structured-zstd vs C FFI"
tool: customSmallerIsBetter
output-file-path: benchmark-results.json
github-token: ${{ steps.bot-token.outputs.token }}
auto-push: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
comment-on-alert: true
alert-comment-cc-users: "@polaz"
alert-threshold: "115%"
fail-threshold: "125%"
fail-on-alert: ${{ github.event_name == 'pull_request' }}
save-data-file: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
benchmark-data-dir-path: dev/bench
40 changes: 40 additions & 0 deletions .github/workflows/cleanup-branches.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Cleanup release-plz branches

on:
schedule:
- cron: "23 3 * * *"
workflow_dispatch:

permissions:
contents: write

jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- name: Delete stale release-plz branches
uses: actions/github-script@v7
with:
script: |
const branches = await github.paginate(
github.rest.repos.listBranches,
{ owner: context.repo.owner, repo: context.repo.repo, per_page: 100 }
);
const stale = branches
.filter(b => b.name.startsWith("release-plz-"))
.map(b => b.name);
stale.sort().reverse();
const toDelete = stale.slice(1);
for (const branch of toDelete) {
try {
await github.rest.git.deleteRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: `heads/${branch}`
});
core.info(`Deleted: ${branch}`);
} catch (e) {
core.info(`Skip ${branch}: ${e.message}`);
}
}
core.info(`Cleaned ${toDelete.length} of ${stale.length} release-plz branches`);
41 changes: 41 additions & 0 deletions .github/workflows/dependabot-auto-merge.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Dependabot Auto-Merge

on:
pull_request:
types: [opened, synchronize, reopened]

permissions:
pull-requests: write
contents: write

jobs:
auto-merge:
runs-on: ubuntu-latest
if: github.event.pull_request.user.login == 'dependabot[bot]'

steps:
- name: Checkout
uses: actions/checkout@v6
Comment thread
polaz marked this conversation as resolved.

- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v2
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"

- name: Generate App Token
id: app-token
if: steps.metadata.outputs.update-type == 'version-update:semver-minor' || steps.metadata.outputs.update-type == 'version-update:semver-patch'
uses: actions/create-github-app-token@v3
with:
app-id: ${{ secrets.RELEASER_APP_ID }}
private-key: ${{ secrets.RELEASER_APP_PRIVATE_KEY }}

- name: Approve and enable auto-merge for minor/patch updates
if: steps.metadata.outputs.update-type == 'version-update:semver-minor' || steps.metadata.outputs.update-type == 'version-update:semver-patch'
run: |
gh pr review --approve "$PR_NUMBER"
gh pr merge --auto --squash "$PR_NUMBER"
Comment thread
polaz marked this conversation as resolved.
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
Comment thread
polaz marked this conversation as resolved.
Loading
Loading