forked from KillingSpark/zstd-rs
-
Notifications
You must be signed in to change notification settings - Fork 0
feat: managed fork setup — README, FUNDING, crate rename to structured-zstd #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
fd9e3e5
docs(readme): managed fork header, donate QR, FUNDING.yml
polaz b555217
chore(crate): rename to structured-zstd v0.0.1, update metadata
polaz eac7e99
ci: add CI matrix, release-plz, codecov, nextest, coderabbit
polaz 742d249
test(bench): add FFI comparison benchmark, fix crate rename references
polaz 33af678
docs(readme): fix import paths to structured_zstd, absolute QR URL, f…
polaz 1caba3f
ci(bench): github-action-benchmark with 5 decompress/compress variations
polaz d2faa13
chore(license): fill Apache-2.0 copyright placeholder
polaz 1e825e1
fix(ci): replace --all-features with explicit features, exclude rustc…
polaz f897b6b
fix(docs): update include_str path after Readme.md → README.md rename
polaz a72459f
fix(dict): isize → i64 in Karp-Rabin hash for 32-bit targets
polaz a3ee043
test(integrity): add 5000-iteration roundtrip integrity tests
polaz 28c7bc7
fix(fork): retain upstream MIT notice, fix bench black_box, pipefail
polaz 75ef90c
fix(lint): suppress dead_code warnings on test-only roundtrip helpers
polaz d4fcdce
fix(docs): replace ruzstd:: with structured_zstd:: in all doc examples
polaz 29ecea3
fix(dict): correct Karp-Rabin h precompute loop and signed modulo
polaz 8c7e646
fix(dict): guard empty pattern in estimate_frequency, pin QR URL to main
polaz 9553c11
fix(ci): add -p structured-zstd for workspace feature resolution, upd…
polaz be3dc74
fix(ci): set working-directory to ruzstd/ for nextest (corpus file pa…
polaz 4206820
refactor(workspace): rename ruzstd/ → zstd/, purge all ruzstd references
polaz 98677f3
fix(dict): fix occurances → occurrences typo in dictionary module
polaz c9d941e
fix(cli): update homepage, repository, license to match fork
polaz c60f020
fix(bench): symmetric per-iteration allocation in decompress benchmark
polaz 412dce8
fix(test): mark corpus files as binary to prevent CRLF conversion on …
polaz 28b0d75
fix(test): use recursive glob in .gitattributes for dict_tests subdirs
polaz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json | ||
|
|
||
| reviews: | ||
| auto_apply_labels: true | ||
| profile: assertive |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| [store] | ||
| dir = "target/nextest" | ||
|
|
||
| [profile.ci] | ||
| retries = 2 | ||
| fail-fast = false | ||
| slow-timeout = { period = "120s", terminate-after = 2 } | ||
|
|
||
| [profile.ci.junit] | ||
| path = "junit.xml" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| github: structured-world | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # GitHub Copilot Instructions for structured-zstd | ||
|
|
||
| ## Project Overview | ||
|
|
||
| Pure Rust zstd implementation — managed fork of [ruzstd (KillingSpark/zstd-rs)](https://github.com/KillingSpark/zstd-rs). Focus: dictionary compression improvements and performance parity with C zstd for CoordiNode LSM-tree. | ||
|
|
||
| ## Review Scope Rules | ||
|
|
||
| **Review ONLY code within the PR's diff.** For issues found outside the diff, suggest creating a separate issue. | ||
|
|
||
| ## Rust Code Standards | ||
|
|
||
| - **Clippy:** Must pass `cargo clippy --all-features -- -D warnings` | ||
| - This is a fork — avoid suggesting architectural changes that diverge too far from upstream | ||
| - Performance-critical code: benchmark before/after any changes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| version: 2 | ||
| updates: | ||
| - package-ecosystem: "cargo" | ||
| directory: "/" | ||
| schedule: | ||
| interval: "weekly" | ||
| open-pull-requests-limit: 10 | ||
| groups: | ||
| minor-and-patch: | ||
| patterns: | ||
| - "*" | ||
| update-types: | ||
| - "minor" | ||
| - "patch" | ||
| commit-message: | ||
| prefix: "chore(deps)" | ||
| - package-ecosystem: "github-actions" | ||
| directory: "/" | ||
| schedule: | ||
| interval: "weekly" | ||
| commit-message: | ||
| prefix: "ci(deps)" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| #!/bin/bash | ||
| # Run compare_ffi benchmarks and produce github-action-benchmark JSON. | ||
| # Output: benchmark-results.json (customSmallerIsBetter format — lower time = better) | ||
| set -eo pipefail | ||
|
|
||
| echo "Running benchmarks..." >&2 | ||
|
|
||
| # Run criterion benchmarks, capture output | ||
| cargo bench --bench compare_ffi -p structured-zstd -- --output-format bencher | tee /tmp/bench-raw.txt | ||
|
|
||
| echo "Parsing results..." >&2 | ||
|
|
||
| # Parse criterion bencher output into github-action-benchmark JSON | ||
| # Format: "test <name> ... bench: <ns> ns/iter (+/- <variance>)" | ||
| python3 - <<'PYEOF' | ||
| import json, re, sys | ||
|
|
||
| results = [] | ||
| with open("/tmp/bench-raw.txt") as f: | ||
| for line in f: | ||
| m = re.match(r"test (\S+)\s+\.\.\. bench:\s+([\d,]+) ns/iter", line) | ||
| if m: | ||
| name = m.group(1) | ||
| ns = int(m.group(2).replace(",", "")) | ||
| # Convert ns to ms for readability | ||
| ms = ns / 1_000_000 | ||
| results.append({ | ||
| "name": name, | ||
| "unit": "ms", | ||
| "value": round(ms, 3), | ||
| }) | ||
|
|
||
| if not results: | ||
| print("ERROR: No benchmark results parsed!", file=sys.stderr) | ||
| sys.exit(1) | ||
|
|
||
| with open("benchmark-results.json", "w") as f: | ||
| json.dump(results, f, indent=2) | ||
|
|
||
| print(f"Wrote {len(results)} benchmark results to benchmark-results.json", file=sys.stderr) | ||
| for r in results: | ||
| print(f" {r['name']}: {r['value']} {r['unit']}", file=sys.stderr) | ||
| PYEOF |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,48 +1,149 @@ | ||
| on: [push, pull_request] | ||
|
|
||
| name: CI | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
| branches: [main] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
|
|
||
| jobs: | ||
| tests: | ||
| name: stable lint, test, build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout sources | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Install stable toolchain | ||
| uses: dtolnay/rust-toolchain@stable | ||
| with: | ||
| components: clippy | ||
| - name: Install cargo-hack and cargo-msrv | ||
| uses: taiki-e/install-action@v2 | ||
| with: | ||
| tool: cargo-hack, cargo-msrv | ||
| - name: check | ||
| run: cargo hack check --workspace --feature-powerset --exclude-features rustc-dep-of-std | ||
| - name: cargo hack clippy | ||
| run: cargo hack clippy --workspace --feature-powerset --exclude-features rustc-dep-of-std | ||
| - name: cargo hack test | ||
| run: cargo hack test --workspace --feature-powerset --exclude-features rustc-dep-of-std | ||
| - name: Verify MSRV (cli) | ||
| run: cargo msrv verify --path cli/ | ||
| - name: Verify MSRV (lib) | ||
| run: cargo msrv verify --path ruzstd/ | ||
|
|
||
| nightly-stuff: | ||
| name: nightly lint, miri | ||
| lint: | ||
| timeout-minutes: 10 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout sources | ||
| uses: actions/checkout@v4 | ||
| - uses: actions/checkout@v6 | ||
| - uses: dtolnay/rust-toolchain@stable | ||
| with: | ||
| components: rustfmt, clippy | ||
| - uses: Swatinem/rust-cache@v2 | ||
| - name: Format | ||
| run: cargo fmt --all -- --check | ||
| - name: Clippy | ||
| run: cargo clippy -p structured-zstd --features hash,std,dict_builder -- -D warnings | ||
|
|
||
| test: | ||
| needs: lint | ||
| timeout-minutes: 15 | ||
| strategy: | ||
| fail-fast: true | ||
| matrix: | ||
| os: [ubuntu-latest, windows-latest, macos-latest] | ||
| runs-on: ${{ matrix.os }} | ||
|
polaz marked this conversation as resolved.
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: dtolnay/rust-toolchain@stable | ||
| - uses: taiki-e/install-action@nextest | ||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| prefix-key: ${{ runner.os }}-cargo | ||
| - name: Test | ||
| working-directory: zstd | ||
| run: cargo nextest run --profile ci -p structured-zstd --features hash,std,dict_builder | ||
| - name: Doc tests | ||
|
polaz marked this conversation as resolved.
polaz marked this conversation as resolved.
|
||
| run: cargo test --doc -p structured-zstd --features hash,std,dict_builder | ||
|
|
||
| - name: Install nightly toolchain | ||
| uses: dtolnay/rust-toolchain@nightly | ||
| cross-i686: | ||
| needs: lint | ||
| timeout-minutes: 15 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: dtolnay/rust-toolchain@stable | ||
| with: | ||
| targets: i686-unknown-linux-gnu | ||
| - uses: taiki-e/install-action@nextest | ||
| - name: Install 32-bit libs | ||
| run: sudo apt-get update && sudo apt-get install -y gcc-multilib | ||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| components: rustfmt, clippy, miri | ||
| prefix-key: cross-i686 | ||
| - name: Test (i686) | ||
| working-directory: zstd | ||
| run: cargo nextest run --profile ci -p structured-zstd --features hash,std,dict_builder --target i686-unknown-linux-gnu | ||
|
|
||
| - run: cargo +nightly fmt --all -- --check | ||
| - run: cargo +nightly clippy --workspace --no-default-features -- -D warnings | ||
| - run: cargo +nightly clippy --workspace -- -D warnings | ||
| - run: cargo +nightly miri test ringbuffer | ||
| - run: cargo +nightly miri test short_Writer | ||
| msrv: | ||
| needs: lint | ||
| timeout-minutes: 15 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: dtolnay/rust-toolchain@stable | ||
| with: | ||
| toolchain: "1.87" | ||
| - uses: taiki-e/install-action@nextest | ||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| prefix-key: msrv | ||
| - name: Test (MSRV) | ||
| working-directory: zstd | ||
| run: cargo nextest run --profile ci -p structured-zstd --features hash,std,dict_builder | ||
|
|
||
| codecov: | ||
| needs: lint | ||
| timeout-minutes: 15 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: dtolnay/rust-toolchain@nightly | ||
| with: | ||
| components: llvm-tools-preview | ||
| - uses: Swatinem/rust-cache@v2 | ||
| - uses: taiki-e/install-action@cargo-llvm-cov | ||
| - run: cargo llvm-cov -p structured-zstd --features hash,std,dict_builder --lcov --output-path lcov.info | ||
| working-directory: zstd | ||
| - uses: codecov/codecov-action@v5 | ||
| with: | ||
| files: zstd/lcov.info | ||
| env: | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
|
|
||
| benchmark: | ||
| name: Performance regression check | ||
| needs: lint | ||
| timeout-minutes: 30 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Generate bot token | ||
| id: bot-token | ||
| if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository | ||
| uses: actions/create-github-app-token@v3 | ||
| with: | ||
| app-id: ${{ secrets.RELEASER_APP_ID }} | ||
| private-key: ${{ secrets.RELEASER_APP_PRIVATE_KEY }} | ||
|
|
||
| - uses: dtolnay/rust-toolchain@stable | ||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| prefix-key: bench | ||
|
|
||
| - name: Run benchmarks | ||
| run: bash .github/scripts/run-benchmarks.sh | ||
|
|
||
| - name: Store benchmark results | ||
| if: steps.bot-token.outputs.token != '' | ||
| uses: benchmark-action/github-action-benchmark@v1 | ||
| with: | ||
| name: "structured-zstd vs C FFI" | ||
| tool: customSmallerIsBetter | ||
| output-file-path: benchmark-results.json | ||
| github-token: ${{ steps.bot-token.outputs.token }} | ||
| auto-push: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} | ||
| comment-on-alert: true | ||
| alert-comment-cc-users: "@polaz" | ||
| alert-threshold: "115%" | ||
| fail-threshold: "125%" | ||
| fail-on-alert: ${{ github.event_name == 'pull_request' }} | ||
| save-data-file: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} | ||
| benchmark-data-dir-path: dev/bench | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: Cleanup release-plz branches | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: "23 3 * * *" | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| jobs: | ||
| cleanup: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Delete stale release-plz branches | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const branches = await github.paginate( | ||
| github.rest.repos.listBranches, | ||
| { owner: context.repo.owner, repo: context.repo.repo, per_page: 100 } | ||
| ); | ||
| const stale = branches | ||
| .filter(b => b.name.startsWith("release-plz-")) | ||
| .map(b => b.name); | ||
| stale.sort().reverse(); | ||
| const toDelete = stale.slice(1); | ||
| for (const branch of toDelete) { | ||
| try { | ||
| await github.rest.git.deleteRef({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| ref: `heads/${branch}` | ||
| }); | ||
| core.info(`Deleted: ${branch}`); | ||
| } catch (e) { | ||
| core.info(`Skip ${branch}: ${e.message}`); | ||
| } | ||
| } | ||
| core.info(`Cleaned ${toDelete.length} of ${stale.length} release-plz branches`); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| name: Dependabot Auto-Merge | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| pull-requests: write | ||
| contents: write | ||
|
|
||
| jobs: | ||
| auto-merge: | ||
| runs-on: ubuntu-latest | ||
| if: github.event.pull_request.user.login == 'dependabot[bot]' | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v6 | ||
|
polaz marked this conversation as resolved.
|
||
|
|
||
| - name: Dependabot metadata | ||
| id: metadata | ||
| uses: dependabot/fetch-metadata@v2 | ||
| with: | ||
| github-token: "${{ secrets.GITHUB_TOKEN }}" | ||
|
|
||
| - name: Generate App Token | ||
| id: app-token | ||
| if: steps.metadata.outputs.update-type == 'version-update:semver-minor' || steps.metadata.outputs.update-type == 'version-update:semver-patch' | ||
| uses: actions/create-github-app-token@v3 | ||
| with: | ||
| app-id: ${{ secrets.RELEASER_APP_ID }} | ||
| private-key: ${{ secrets.RELEASER_APP_PRIVATE_KEY }} | ||
|
|
||
| - name: Approve and enable auto-merge for minor/patch updates | ||
| if: steps.metadata.outputs.update-type == 'version-update:semver-minor' || steps.metadata.outputs.update-type == 'version-update:semver-patch' | ||
| run: | | ||
| gh pr review --approve "$PR_NUMBER" | ||
| gh pr merge --auto --squash "$PR_NUMBER" | ||
|
polaz marked this conversation as resolved.
|
||
| env: | ||
| GH_TOKEN: ${{ steps.app-token.outputs.token }} | ||
| PR_NUMBER: ${{ github.event.pull_request.number }} | ||
|
polaz marked this conversation as resolved.
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.