Test infrastructure, compatibility tests, and architecture improvements

[unclesp1d3r]

Summary

• Add comprehensive integration test suites for evaluator, MIME, tags, property tests, and end-to-end flows (~940 tests total)
• Add benchmark infrastructure (parser, evaluation, I/O) with CI workflow for regression tracking
• Fix CI workflows for benchmarks and compatibility tests
• Refactor architecture based on review: eliminate redundant work in evaluator, fix lossy error conversion, add proper error variants, cross-type integer coercion, deduplicate result-building logic, cache TagExtractor
• Optimize evaluation pipeline: SIMD null scanning via memchr, remove redundant bounds checks, release profile with LTO
• Simplify code: idiomatic then_some(), reuse build_result, remove library eprintln calls
• Split parser module into focused submodules (format, hierarchy, loader, preprocessing)

Test plan

• just ci-check passes (940 tests, clippy, fmt, cargo check)
• All pre-commit hooks pass
• Benchmarks compile and run
• Doc tests pass
• CI workflows pass on GitHub Actions

🤖 Generated with Claude Code

[coderabbitai]

Caution

Review failed

Failed to post review comments

Summary by CodeRabbit

• New Features

  • CI benchmarks (scheduled/PR/manual) and many local benchmark targets; large new integration, property, and unit test suites.

• Documentation

  • Expanded testing docs: benchmarks, benchmark CI comparisons, fuzzing, compatibility, and developer patterns.

• Performance

  • Faster evaluation with periodic timeout checks and cached MIME lookups.

• API Changes

  • Evaluation APIs now borrow configuration and single-rule checks return optional match payloads; new error variants for config/file issues.

• UX / CLI

  • Streamlined JSON output, clearer load-error messages, and distinct exit codes for config/file failures.

Walkthrough

Splits the parser into format/preprocessing/hierarchy/loader modules, adds Criterion benchmarks and a Benchmarks CI workflow with PR-vs-main comparison, changes evaluator APIs to Option-based matches and borrowed EvaluationConfig, adds MIME caching and output conversion helpers, and introduces extensive tests, docs, and tooling updates.

Changes

Cohort / File(s)	Summary
Workflows & CI \.github/workflows/benchmarks.yml, \.github/workflows/compatibility.yml	Adds a scheduled/manual/PR Benchmarks workflow with PR-vs-main comparison and a compatibility workflow matrix with caching, verification, and artifact uploads.
Benchmarks & Manifest benches/parser_bench.rs, benches/evaluation_bench.rs, benches/io_bench.rs, Cargo.toml	Adds three Criterion benchmark suites, registers benches in Cargo.toml, bumps dev-deps, and adjusts release/dist profiles.
Parser refactor & new modules src/parser/mod.rs, src/parser/format.rs, src/parser/preprocessing.rs, src/parser/hierarchy.rs, src/parser/loader.rs	Reorganizes parser into new modules (format, preprocessing, hierarchy, loader), moves/exports detect_format/load_magic_file/load_magic_directory/parse_text_magic_file, implements line preprocessing and hierarchy builder, and adds comprehensive tests.
Evaluator & types src/evaluator/mod.rs, src/evaluator/operators.rs, src/evaluator/types.rs	Changes evaluate_single_rule to return Option<(offset, Value)>, evaluate_rules_with_config now takes &EvaluationConfig and uses synchronous evaluation with periodic timeout checks; adds cross-type integer coercion and memchr-based string reads.
Library surface & errors src/lib.rs, src/error.rs	Adds LibmagicError::ConfigError{reason} and LibmagicError::FileError(String); MagicDatabase now caches a mime_mapper field and initializers updated; From<IoError> maps to FileError.
Output & CLI src/output/mod.rs, src/main.rs	Adds conversion helpers (from_evaluator_match, from_library_result) and a lazy TagExtractor; CLI uses EvaluationResult::from_library_result and maps new error variants to exit codes.
Parser helpers & loader tests src/parser/..., tests/*	Adds LineInfo, preprocess_lines, parse_magic_rule_line, deterministic directory loader, and many unit tests for parsing/loader edge cases and error paths.
Tests (property, integration, unit, mime, tags) tests/property_tests.rs, tests/integration_tests.rs, tests/evaluator_tests.rs, tests/mime_tests.rs, tests/tags_tests.rs, tests/compatibility_tests.rs	Adds property-based tests (proptest), broad integration suites, evaluator/mime/tag tests, and updates compatibility tests to use --use-builtin.
Parser detection src/parser/format.rs	Adds MagicFileFormat enum and detect_format(Path) detecting Text/Directory/Binary formats with unit tests.
Docs & Tooling docs/src/testing.md, docs/src/evaluator.md, docs/src/getting-started.md, justfile, mise.toml, .gitignore	Docs updated to document benchmarks, testing, and evaluate_rules signature changes (&EvaluationConfig); justfile adds coverage-report/summary targets; mise adds tools; .gitignore updated.
Internal guidance .claude/..., .claude/skills/SKILL.md	Adds multiple project guidance documents (AST derive, error-handling, testing conventions, co-change awareness, strict Clippy, just usage, skills).

Sequence Diagram(s)

sequenceDiagram
    participant PR as Pull Request
    participant GH as GitHub Actions
    participant Runner as Runner (ubuntu-latest)
    participant CheckoutPR as Checkout PR
    participant Setup as Setup Tools & Cache
    participant BenchPR as Run benches (baseline=pr)
    participant UploadPR as Upload PR artifact
    participant CheckoutMain as Checkout main
    participant BenchMain as Run benches (baseline=main)
    participant Critcmp as critcmp Compare & Report

    PR->>GH: trigger (PR update / manual / schedule)
    GH->>Runner: start benchmark-compare job
    Runner->>CheckoutPR: checkout PR branch (fetch-depth: 0)
    Runner->>Setup: install tools (mise), restore caches
    Runner->>BenchPR: run `cargo bench` (save baseline=pr)
    BenchPR->>UploadPR: upload `target/criterion/` (pr)
    Runner->>CheckoutMain: checkout main branch
    Runner->>Setup: ensure tools & caches
    Runner->>BenchMain: run `cargo bench` (save baseline=main)
    BenchMain->>Critcmp: run `critcmp` to compare baselines
    Critcmp->>GH: append formatted comparison to PR summary

Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

• feat: evaluation enhancements with confidence, MIME, tags, metadata #29 — overlaps output conversion, EvaluationResult metadata, and MIME/tag enrichment changes.
• 12 add magic file format detection subtask 143 #25 — related to parser file-format detection and loader API refactor.
• feat: cli enhancements multiple files stdin magic discovery #27 — related to evaluator API changes and timeout/threading modifications.

Poem

🐇 I hopped through benches at Sunday dawn,

I stacked the rules and kept the errors gone,
MIME maps cached, timeouts checked in line,
Tests sprouted carrots, parser roots align,
CI hums softly — cozy, quick, and fine.

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The pull request title accurately summarizes the main changes: test infrastructure additions, compatibility test improvements, and architecture refactoring with clear focus on the three primary areas of work.
Description check	✅ Passed	The pull request description is directly related to the changeset, providing a well-organized summary of test additions, benchmark infrastructure, CI fixes, architectural refactoring, performance optimizations, and code cleanups with a clear test plan.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch 22-test-infrastructure-compatibility-tests-coverage

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

❌ Patch coverage is 94.30315% with 56 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/parser/loader.rs	92.48%	16 Missing ⚠️
src/lib.rs	78.57%	12 Missing ⚠️
src/parser/preprocessing.rs	97.11%	9 Missing ⚠️
src/main.rs	40.00%	6 Missing ⚠️
src/evaluator/operators.rs	89.18%	4 Missing ⚠️
src/output/mod.rs	93.33%	3 Missing ⚠️
src/parser/format.rs	97.14%	3 Missing ⚠️
src/evaluator/mod.rs	96.87%	2 Missing ⚠️
src/parser/hierarchy.rs	99.25%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!