feat: Create rmagic cli

[unclesp1d3r]

This pull request introduces compatibility testing infrastructure, updates CI/CD workflows, and refines error handling and test file management for the project. The most significant changes include adding a GitHub Actions workflow for compatibility tests, updating the error handling model to use more structured error types, and marking key implementation milestones as complete. The changes also improve the handling of test and result files across the codebase and update development dependencies to support enhanced testing.

Compatibility Testing Infrastructure

• Added .github/workflows/compatibility.yml to run compatibility tests across Ubuntu, Windows, and macOS, including scheduled nightly runs and verification of compatibility test files.
• Updated justfile with new recipes for verifying compatibility test files and running compatibility tests, including platform-specific checks and integration with CI. Added ci-check-compatibility for full CI parity including compatibility tests. [1] [2]

Error Handling Improvements

• Refactored error types in src/lib.rs to use the new LibmagicError, EvaluationError, and ParseError enums from src/error.rs, replacing previous string-based error handling. Improved conversion from IO errors and updated error propagation in offset resolution logic. [1] [2] [3]
• Enhanced error propagation and test assertions in src/evaluator/offset.rs to use structured error enums, improving clarity and robustness of error handling in offset resolution and related tests. [1] [2] [3]

Project Management and Documentation

• Marked major implementation milestones as completed in .kiro/specs/rust-libmagic-implementation/tasks.md, including CLI argument structure, file processing, JSON output, string type support, and error types.

Test and Result File Management

• Updated .gitattributes, .prettierignore, .mdformat.toml to treat *.result and *.testfile as binary files and exclude them from formatting and prettification. [1] [2] [3]
• Added minimal magic files for testing and CI/CD environments: missing.magic and nonexistent.magic. [1] [2]

Development Dependency Updates

• Added insta, tempfile, and updated nix as dev-dependencies in Cargo.toml to support snapshot testing and cross-platform file operations.

These changes collectively improve the project's reliability, test coverage, and maintainability, especially in cross-platform and CI/CD scenarios.

[coderabbitai]

Caution

Review failed

Failed to post review comments

Summary by CodeRabbit

• New Features

  • JSON output (--json) with structured formatting and hex value support; new masked bitwise matching and improved string reading (UTF‑8, null‑terminated).
  • CLI revamped with confirmable output modes and platform-aware magic-file resolution.

• Testing

  • Large cross‑platform compatibility and CLI integration suites added; normalization helpers and CI job to run compatibility tests across OSes (scheduled and on PRs).

• Improvements

  • Graceful per‑rule degradation, richer error reporting, and updated tooling/format ignore patterns.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

Walkthrough

Adds compatibility CI/workflow and many libmagic test fixtures; introduces a public error module and structured errors; updates evaluator to gracefully skip non‑fatal rule errors and add string reading and BitwiseAndMask operator; implements clap-derive CLI with JSON/text output and JSON formatting; extensive tests and docs added/updated.

Changes

Cohort / File(s)	Summary
CI & Automation **/.github/workflows/compatibility.yml**, **/.github/workflows/ci.yml**, **/justfile**, **/.kiro/hooks/ci-auto-fix.kiro.hook**, **/.coderabbitai.yaml**, **/.pre-commit-config.yaml**	Add compatibility workflow and CI targets, update Codecov slug, narrow KiRo hook triggers, exclude third_party from code-review filter, and comment out mdformat hook.
Repo ignores & attributes /.gitattributes, /.mdformat.toml, /.prettierignore, /.gitignore	Treat **/*.result and **/*.testfile as binary and add them to formatter/Prettier ignores; remove test_files/ from .gitignore.
Errors & exports src/error.rs, src/lib.rs	New public error module (LibmagicError, ParseError, EvaluationError) with From impls/constructors; re-export errors and add IoError conversion.
Evaluator & types src/evaluator/mod.rs, src/evaluator/offset.rs, src/evaluator/types.rs, src/evaluator/operators.rs	Add evaluate_rules_with_config; map offset/type errors to structured EvaluationError; make per-rule failures non‑fatal (log-and-skip) except critical errors; add read_string and integrate into read_typed_value; implement BitwiseAndMask operator.
Parser & AST src/parser/grammar.rs, src/parser/ast.rs	Expose parsing helpers (types, offsets, messages, full rules), support mixed hex/ASCII parsing; add Operator::BitwiseAndMask(u64) AST variant.
CLI & main src/main.rs	Replace manual parsing with clap-derive Args and OutputFormat; implement argument validation, platform-aware magic-file resolution with CI/test fallback, centralized run_analysis and error mapping; add extensive CLI tests.
Output (JSON) src/output/json.rs, src/output/mod.rs	New JsonMatchResult/JsonOutput types, converters, hex formatting, pretty/compact JSON formatters; export pub mod json; add debug-only confidence validation.
Tests & test utils tests/*, tests/common/mod.rs, tests/compatibility/README.md	Add compatibility runner, CLI/JSON integration and normalization tests, insta snapshots, test helpers, and README.
Third‑party fixtures & notices third_party/COPYING, third_party/NOTICE.md, third_party/tests/*	Add many canonical libmagic test fixtures (*.testfile, *.result, .magic, .flags, README) and license/notice files for compatibility testing.
Small CI/test magic files missing.magic, nonexistent.magic	Add minimal magic rule files used as CI/test fallbacks.
Cargo / dev deps Cargo.toml	Add dev-dependencies: insta (json feature), regex, temp-env, tempfile.
Docs docs/src/*.md	Update docs to reference third_party paths, add CLI testing/snapshot guidance, and minor formatting/troubleshooting edits.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant CLI as rmagic (CLI)
  participant Args as Args/Clap
  participant FS as Filesystem
  participant DB as MagicDatabase
  participant Eval as Evaluator
  participant Out as Output(JSON/Text)

  User->>CLI: Invoke with [--json|--text] --file <path> [--magic-file]
  CLI->>Args: Parse arguments
  Args-->>CLI: Args{file, format, magic_path?}
  CLI->>FS: Validate input file / magic file (resolve defaults)
  alt Magic file missing (CI/test fallback)
    CLI->>FS: Create/download basic magic file
  end
  CLI->>DB: load_from_file(magic_path)
  DB-->>CLI: rules/config
  CLI->>Eval: evaluate_rules_with_config(rules, buffer, config)
  alt Non-critical errors
    Eval-->>Eval: Log & skip rule errors
  else Critical (timeout/recursion)
    Eval-->>CLI: Propagate LibmagicError
  end
  Eval-->>CLI: Vec<MatchResult>
  CLI->>Out: Render JSON or Text
  Out-->>User: Output

Loading

sequenceDiagram
  autonumber
  participant Eval as Evaluator
  participant Rule as Rule Walker
  participant Types as Type Reader
  participant Err as Error Module

  Eval->>Rule: Iterate rules
  Rule->>Rule: Resolve offsets
  alt Offset errors
    Rule->>Err: Map to EvaluationError::{BufferOverrun|InvalidOffset}
    Rule-->>Eval: Skip rule (log)
  else OK
    Rule->>Types: read_typed_value (incl. read_string)
    alt Type read error (non-critical)
      Types->>Err: TypeReadError -> EvaluationError
      Rule-->>Eval: Skip match (log)
    else Match + children
      Rule->>Rule: Evaluate children (depth++)
      alt Critical (Timeout/RecursionLimitExceeded)
        Rule-->>Eval: Propagate error
      else Non-critical child errors
        Rule-->>Eval: Skip child (log)
      end
    end
  end
  Eval-->>Caller: Results or error

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~70 minutes

Files/areas needing extra attention:

• src/error.rs and From conversions (variant shapes and Display messages).
• src/evaluator/* (graceful-degradation semantics, recursion/timeout propagation, and tests).
• src/main.rs (argument parsing, magic-file resolution, CI/test fallback code paths).
• tests/compatibility and the large addition of third_party fixtures (test harness expectations and licensing notices).

Possibly related issues

• CLI: Implement output format selection (--top, --json flags) Stringy#31 — Adds requested --json/format handling and JSON output; this PR implements JSON output and CLI integration that address that issue.

Possibly related PRs

• feat: Add evaluation engine with offset resolution and CI/CD automation #5 — Overlaps evaluator, offset resolution, operator/type-read logic and related modules; strong code-level relation.
• feat: add development infrastructure, automation hooks, and comprehensive documentation #4 — Touches KiRo hook triggers and CI tooling similar to .kiro/hooks/ci-auto-fix.kiro.hook changes.

Poem

Hop hop, I parse and peek,
Through magic bytes I softly sneak.
Errors softened, rules let pass,
JSON shines in every glass.
Tests and CI clap their paws — hooray for this small rabbit's bash! 🥕✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The PR title 'feat: Create rmagic cli' is only partially related to the changeset. The changeset includes significant compatibility testing infrastructure, error handling refactoring, test file management, and CI/CD workflow updates. While a CLI is part of these changes, the title does not capture the main scope of the work.	Consider a more comprehensive title that reflects the major changes: 'feat: Add rmagic CLI with compatibility testing infrastructure and error handling refactoring' or similar, to better convey the full scope of the PR.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description check	✅ Passed	The PR description is well-written and thoroughly documents all major changes including compatibility testing infrastructure, error handling improvements, test file management, and dependency updates. It clearly relates to the actual changeset across multiple subsystems.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch create_rmagic_cli

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Copilot reviewed 176 out of 225 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 176 out of 225 changed files in this pull request and generated no new comments.

[unclesp1d3r]

@copilot The test-cross-platform windows test is failing in the CI chain. Please examine the failures here https://github.com/EvilBit-Labs/libmagic-rs/actions/runs/20216079101/job/58029213726?pr=7 and fix the issues. Run just ci-check to ensure everything still works before commit your fixes.

[Copilot]

@unclesp1d3r I've opened a new pull request, #10, to work on those changes. Once the pull request is ready, I'll request review from you.

[codecov]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment

Thanks for integrating Codecov - We've got you covered ☂️