Skip to content

refactor(tests): extract CLI tests from main.rs into integration tests#159

Merged
unclesp1d3r merged 9 commits into
mainfrom
61-refactor-extract-cli-tests-from-mainrs-into-integration-tests
Mar 7, 2026
Merged

refactor(tests): extract CLI tests from main.rs into integration tests#159
unclesp1d3r merged 9 commits into
mainfrom
61-refactor-extract-cli-tests-from-mainrs-into-integration-tests

Conversation

@unclesp1d3r
Copy link
Copy Markdown
Member

Summary

  • Extracts CLI unit tests from src/main.rs (~430 lines removed) into a dedicated tests/cli_integration.rs integration test file (~887 lines)
  • Uses assert_cmd crate for proper subprocess-based CLI testing instead of calling internal functions directly
  • Adds assert_cmd, predicates, and tempfile as dev dependencies for robust test infrastructure
  • Updates testing documentation in docs/src/testing.md to reflect the new integration test structure

Test Plan

  • All 1,188 tests pass (cargo test)
  • CodeRabbit review: no findings
  • CI passes (clippy, fmt, tests, audit)
  • Verify CLI integration tests exercise all major code paths (ELF, PNG, JPEG, PDF, ZIP, GIF detection)
  • Verify JSON output format tests pass
  • Verify strict mode tests pass

Closes #61

🤖 Generated with Claude Code

unclesp1d3r and others added 4 commits March 6, 2026 17:54
…ess testing

This commit introduces a new set of integration tests for the `rmagic` CLI, leveraging the `assert_cmd` crate to ensure reliable subprocess execution and output validation. The tests cover various file types (ELF, PNG, JPEG, PDF, ZIP, GIF) and include checks for both standard and strict modes, as well as JSON output formatting.

- Created `tests/cli_integration.rs` to house the new integration tests.
- Implemented helper functions for generating temporary test files.
- Enhanced test coverage for CLI argument parsing and output verification.

These changes improve the robustness of the CLI testing framework and facilitate easier maintenance and expansion of test cases in the future.

Co-authored-by: dosubot[bot] <131922026+dosubot[bot]@users.noreply.github.com>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
This commit updates the Rust version specification in both `mise.lock` and `mise.toml` files. The `mise.lock` file now specifies Rust version `1.93.1`, while the `mise.toml` file has been changed to use the `latest` version instead of `stable`. These changes ensure that the project is aligned with the latest Rust tooling and dependencies.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
This commit reformats the `dist-workspace.toml` file to enhance readability by adjusting the spacing and organizing the `targets` list into a multi-line format. The changes do not affect the functionality but improve the clarity of the configuration.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
This commit updates several dependencies in the `Cargo.lock` file, including `anyhow` to version `1.0.102`, `bumpalo` to `3.20.2`, `getrandom` to `0.4.2`, `js-sys` to `0.3.91`, `quote` to `1.0.45`, `regex-syntax` to `0.8.10`, `syn` to `2.0.117`, and `wasm-bindgen` related packages to `0.2.114`. Additionally, the `r-efi` dependency has been added with version `6.0.0`. These updates ensure compatibility with the latest features and improvements in the respective libraries.

Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Copilot AI review requested due to automatic review settings March 7, 2026 00:04
@unclesp1d3r unclesp1d3r linked an issue Mar 7, 2026 that may be closed by this pull request
6 tasks
@dosubot dosubot Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Mar 7, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 7, 2026

Caution

Review failed

Pull request was closed or merged during review

Summary by CodeRabbit

  • Tests

    • Reworked CLI test suite to use subprocess-based integration tests with broader end-to-end coverage (stdin, multi-file, formats, timeouts, completion).
    • Replaced fragile snapshot/normalization helpers with isolated, tempfile-based test workflows.
  • Chores

    • Updated several dependencies and added dev testing tools to support new integration tests.
  • Documentation

    • Testing docs updated to describe the new CLI integration approach and how to run the tests.

Walkthrough

Extracts fragile, process-wide CLI tests from src/main.rs into a subprocess-based integration suite tests/cli_integration.rs, adds dev-dependencies for subprocess testing, removes insta-based normalization helpers and snapshots, updates docs and CI/config formatting. No public API changes.

Changes

Cohort / File(s) Summary
Dependency / Manifest
Cargo.toml, dist-workspace.toml, mise.toml
Updated runtime deps (clap_complete, ctrlc, memchr), added serde_json; added dev-deps assert_cmd, predicates, tempfile; minor TOML formatting and Rust tool pin change.
New Integration Tests
tests/cli_integration.rs, tests/compatibility_tests.rs
Adds a comprehensive subprocess-based CLI integration test suite (assert_cmd, predicates, tempfile) and improves binary discovery for compatibility tests.
Removed Legacy Tests & Helpers
tests/cli_integration_tests.rs, tests/cli_normalization.rs, tests/common/mod.rs, tests/snapshots/...
Deletes old insta snapshot tests, normalization helpers, FD-capture scaffolding, and snapshot fixtures.
Unit Tests / Test Scaffolding Moved
src/main.rs
Removed process-wide FD/stdin capture helpers and transplanted CLI-focused tests to integration tests; left unit tests for small pure functions.
Docs & Guidance
docs/src/testing.md, docs/src/testing-guidelines.md, docs/plan/..., src/builtin_rules.rs, .claude/...
Rewrote testing docs to describe subprocess-based CLI integration testing, updated references to new test filename, and adjusted guidance.
Misc
Cargo.toml dev-deps entries, .claude/*
Added/updated CI/dev guidance metadata reflecting test file renames and testing convention changes.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐇
I hopped from main into the test suite glen,
Laid down tempfiles, spawned subprocesses then.
No more fd juggling, no snapshots to comb,
Clean runs and green lights — I’m home, I’m home.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'refactor(tests): extract CLI tests from main.rs into integration tests' clearly and concisely summarizes the primary change: extracting CLI tests from main.rs into a new integration test file.
Description check ✅ Passed The description is related to the changeset, providing a clear summary of extraction work, technical approach (assert_cmd), dependency additions, and documentation updates.
Linked Issues check ✅ Passed The PR successfully meets all acceptance criteria from #61: src/main.rs is reduced from ~1,647 to ~1,167 lines, CLI tests moved to tests/cli_integration.rs using assert_cmd, stdin tests use write_stdin(), FD_MUTEX removed, and test coverage preserved with 1,188 passing tests.
Out of Scope Changes check ✅ Passed All changes are scoped to the refactoring objective: test extraction (main.rs, cli_integration.rs), dev-dependency additions (assert_cmd, predicates, tempfile), documentation updates, and configuration adjustments related to testing infrastructure.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 61-refactor-extract-cli-tests-from-mainrs-into-integration-tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Mar 7, 2026

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 CI must pass

Wonderful, this rule succeeded.

All CI checks must pass. Release-plz PRs are exempt because they only bump versions and changelogs (code was already tested on main), and GITHUB_TOKEN-triggered force-pushes suppress CI.

  • check-success = coverage
  • check-success = quality
  • check-success = test
  • check-success = test-cross-platform (macos-latest, macOS)
  • check-success = test-cross-platform (ubuntu-22.04, Linux)
  • check-success = test-cross-platform (ubuntu-latest, Linux)
  • check-success = test-cross-platform (windows-latest, Windows)

🟢 Do not merge outdated PRs

Wonderful, this rule succeeded.

Make sure PRs are within 10 commits of the base branch before merging

  • #commits-behind <= 3

@unclesp1d3r unclesp1d3r self-assigned this Mar 7, 2026
@dosubot dosubot Bot added cli Command-line interface and tools testing Test infrastructure and coverage labels Mar 7, 2026
@dosubot
Copy link
Copy Markdown
Contributor

dosubot Bot commented Mar 7, 2026

Related Documentation

3 document(s) may need updating based on files changed in this PR:

libMagic-rs

Testing Strategy And Coverage Requirements
View Suggested Changes
@@ -4,7 +4,7 @@
 
 The libmagic-rs project implements a comprehensive, multi-layered testing strategy designed to ensure reliability, maintainability, and cross-platform compatibility for its pure-Rust replacement of libmagic. The testing approach combines traditional unit and integration testing with modern property-based testing techniques, enforcing strict coverage requirements of >85% line coverage.
 
-The project maintains [over 900 test cases](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/) across [22 source files with unit tests](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/) and [11 integration test files](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/), achieving [94%+ test coverage](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/codecov.yml). The testing infrastructure leverages [cargo-nextest for faster test execution](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/mise.toml#L8), [proptest for property-based testing](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/Cargo.toml#L166), and [cargo llvm-cov for coverage measurement](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/mise.toml#L7).
+The project maintains [over 900 test cases](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/) across [22 source files with unit tests](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/) and [12 integration test files](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/), achieving [94%+ test coverage](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/codecov.yml). The testing infrastructure leverages [cargo-nextest for faster test execution](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/mise.toml#L8), [proptest for property-based testing](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/Cargo.toml#L166), and [cargo llvm-cov for coverage measurement](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/mise.toml#L7).
 
 A unique aspect of the testing strategy is the [build script testing pattern](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/build_helpers.rs), which extracts build logic into testable modules using `#[cfg(any(test, doc))]` to enable comprehensive testing of build-time code generation. All testing commands are executed through [mise exec](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/justfile#L12) to ensure consistent toolchain versions, and the [just ci-check command](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/justfile#L207) provides local CI parity for developers.
 
@@ -16,7 +16,7 @@
 
 **Core library tests:**
 - [src/lib.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/lib.rs) - 26 unit tests
-- [src/main.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/main.rs) - 55 unit tests (CLI functionality)
+- [src/main.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/main.rs) - Core CLI logic (tested via integration tests)
 - [src/error.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/error.rs) - 24 unit tests
 - [src/build_helpers.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/build_helpers.rs) - 35 unit tests
 
@@ -31,14 +31,14 @@
 - [src/parser/ast.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/parser/ast.rs) - 36 unit tests
 - [src/parser/preprocessing.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/parser/preprocessing.rs) - 32 unit tests
 
-The unit test count totals approximately [746 tests across all source files](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/).
+The unit test count totals approximately [691 tests across all source files](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/) (following CLI test extraction to integration tests).
 
 ### Integration Tests Directory
 
-The [tests/ directory](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/) contains [11 integration test files](https://github.com/EvilBit-Labs/libmagic-rs/tree/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests) with approximately 155 test functions testing complete workflows:
+The [tests/ directory](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/) contains [12 integration test files](https://github.com/EvilBit-Labs/libmagic-rs/tree/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests) testing complete workflows:
 
 1. **[tests/integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/integration_tests.rs)** (378 lines) - Core end-to-end tests covering built-in rules, file loading, and custom magic rules, including quad type tests (`test_quad_lequad_matches_little_endian_value`, `test_quad_bequad_matches_big_endian_value`, `test_quad_signed_negative_one`, `test_quad_nested_child_rule_with_offset`)
-2. **[tests/cli_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/cli_integration_tests.rs)** (1,625 lines) - Comprehensive CLI testing including canonical libmagic compatibility, error handling, and edge cases
+2. **[tests/cli_integration.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/cli_integration.rs)** (887 lines) - CLI integration tests using `assert_cmd` for subprocess-based testing. This approach provides natural process isolation and eliminates fd manipulation. Tests cover builtin detection (ELF, PNG, JPEG, PDF, ZIP, GIF), stdin handling with truncation warnings, multiple file processing, error handling, timeout validation, output formats (text/JSON), shell completion, custom magic files, and edge cases like Unicode filenames and empty files.
 3. **[tests/property_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/property_tests.rs)** (221 lines) - Property-based testing with proptest
 4. **[tests/evaluator_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/evaluator_tests.rs)** - Evaluator integration tests for confidence calculation and configuration
 5. **[tests/compatibility_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/compatibility_tests.rs)** - Compatibility with original libmagic implementation
@@ -111,7 +111,7 @@
 ### CI Coverage Workflow
 
 The [CI workflow coverage job](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/.github/workflows/ci.yml#L82-L111):
-1. Generates coverage with [`--test-threads=1`](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/.github/workflows/ci.yml#L96) to prevent race conditions in stdin-mocking tests
+1. Generates coverage with standard `cargo test`
 2. Combines reports and outputs lcov.info
 3. Uploads to [Codecov](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/.github/workflows/ci.yml#L101-L107) and [Qlty](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/.github/workflows/ci.yml#L108-L111) services
 
@@ -403,7 +403,7 @@
 | File Path | Description | Lines |
 |-----------|-------------|-------|
 | [tests/integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/integration_tests.rs) | Core end-to-end integration tests | 378 |
-| [tests/cli_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/cli_integration_tests.rs) | Comprehensive CLI testing | 1,625 |
+| [tests/cli_integration.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/cli_integration.rs) | CLI integration tests with assert_cmd | 887 |
 | [tests/property_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/property_tests.rs) | Property-based tests with proptest | 221 |
 | [tests/common/mod.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/tests/common/mod.rs) | Shared test utilities | 185 |
 | [src/build_helpers.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/build_helpers.rs) | Build script testing module | 709 |

✅ Accepted

Testing Strategy And Infrastructure
View Suggested Changes
@@ -1,8 +1,8 @@
 # Testing Strategy and Infrastructure
 
-The [libmagic-rs project maintains a comprehensive, multi-layered testing strategy with over 900 test cases across 22 source files with unit tests and 11 integration test files, achieving 94%+ test coverage](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384). The testing architecture combines unit tests, integration tests, property-based testing with proptest, snapshot testing with insta, CLI testing with assert_cmd, and performance benchmarking with criterion. [All testing is coordinated through cargo-nextest version 0.9.123-b.4 as the primary test runner](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384), with strict coverage requirements enforced through cargo llvm-cov (≥85% overall, >90% for new code).
-
-The infrastructure represents a durable testing architecture designed specifically for a magic file parser and evaluator, featuring unique patterns like build script testing through extracted modules, cross-platform CLI snapshot normalization, and property-based fuzzing of rule evaluation. [Testing tools are version-pinned through mise.toml to ensure consistency between local development and CI environments](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384).
+The [libmagic-rs project maintains a comprehensive, multi-layered testing strategy with over 900 test cases across 22 source files with unit tests and 12 integration test files, achieving 94%+ test coverage](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384). The testing architecture combines unit tests, integration tests, property-based testing with proptest, snapshot testing with insta, CLI testing with assert_cmd, and performance benchmarking with criterion. [All testing is coordinated through cargo-nextest version 0.9.123-b.4 as the primary test runner](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384), with strict coverage requirements enforced through cargo llvm-cov (≥85% overall, >90% for new code).
+
+The infrastructure represents a durable testing architecture designed specifically for a magic file parser and evaluator, featuring unique patterns like build script testing through extracted modules, subprocess-based CLI testing with assert_cmd, cross-platform CLI snapshot normalization, and property-based fuzzing of rule evaluation. [Testing tools are version-pinned through mise.toml to ensure consistency between local development and CI environments](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384).
 
 ## Test Categories and Organization
 
@@ -11,38 +11,40 @@
 [Unit tests are located in `#[cfg(test)]` modules within source files](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384), keeping tests close to the code they validate:
 
 - **[src/lib.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/lib.rs)** - 26 unit tests
-- **[src/main.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/main.rs)** - 55 unit tests (CLI functionality)
+- **[src/main.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/main.rs)** - 4 unit tests
 - **[src/evaluator/mod.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/evaluator/mod.rs)** - 77 unit tests
 - **[src/parser/grammar.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/parser/grammar.rs)** - 91 unit tests
 - **[src/evaluator/operators.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e925ef6b3f2208fc8805a728ba3de55956f4447a/src/evaluator/operators.rs)** - 86 unit tests covering equality, inequality, comparison, and bitwise operations
 
-[Total unit test count: approximately 746 tests across all source files](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384).
+[Total unit test count: approximately 695 tests across all source files](https://app.dosu.dev/documents/62393a9d-87d8-43cd-b7c1-b117edd5a384).
 
 ### Integration Tests Directory
 
-[The tests/ directory contains 11 integration test files](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests) with approximately 155 test functions:
+[The tests/ directory contains 12 integration test files](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests) with approximately 205 test functions:
 
 1. **[tests/integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/integration_tests.rs)** (468 lines) - [Core end-to-end tests covering the complete workflow: load database, evaluate files/buffers, verify output formatting and metadata](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/integration_tests.rs). Tests built-in rules, file loading, and custom magic rules. Includes quad type tests: `test_quad_lequad_matches_little_endian_value`, `test_quad_bequad_matches_big_endian_value`, `test_quad_signed_negative_one`, `test_quad_nested_child_rule_with_offset`.
 
-2. **[tests/cli_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration_tests.rs)** (1,625 lines) - [CLI integration tests using the canonical libmagic test suite](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration_tests.rs). Verifies command-line interface functionality by running against third_party/tests/, covering canonical test suite compatibility, multiple file processing, strict mode (`--strict`), built-in rules (`--use-builtin`), and JSON lines output.
-
-3. **[tests/property_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs)** (225 lines) - [Property-based tests verifying that evaluator never panics, buffer access is bounds-checked, metadata consistency, and serde roundtrips](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs). Includes [6 custom proptest strategies](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs#L18-L84): arb_offset_spec, arb_type_kind, arb_operator, arb_value, arb_magic_rule, arb_buffer.
-
-4. **[tests/evaluator_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/evaluator_tests.rs)** (248 lines) - [Evaluator integration tests through the public MagicDatabase API](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/evaluator_tests.rs). Tests confidence calculation, rule ordering, evaluation behavior, and validates that known file types return non-zero confidence scores.
-
-5. **[tests/compatibility_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility_tests.rs)** (360 lines) - [Compatibility tests ensuring identical results to the original libmagic](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility_tests.rs). [Test files are downloaded from the file/file repository](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility/README.md#L7) and compared against GNU file command.
-
-6. **[tests/parser_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/parser_integration_tests.rs)** (380 lines) - [End-to-end integration tests for magic file parser and database integration](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/parser_integration_tests.rs). Validates the complete flow from file/directory loading through rule evaluation.
-
-7. **[tests/directory_loading_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/directory_loading_tests.rs)** (431 lines) - [Integration tests for directory loading functionality](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/directory_loading_tests.rs). Validates `load_magic_directory()` function with various directory structures and batch loading of magic files from directories.
-
-8. **[tests/mime_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/mime_tests.rs)** (230 lines) - [MIME type mapping integration tests](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/mime_tests.rs). Tests MIME type detection through the public API including hardcoded mappings, prefix matching, and case insensitivity.
-
-9. **[tests/tags_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/tags_tests.rs)** (163 lines) - [Tag extraction integration tests](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/tags_tests.rs). Tests keyword extraction, case insensitivity, rule path tags, and custom keyword support.
-
-10. **[tests/json_integration_test.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/json_integration_test.rs)** (327 lines) - [Integration tests for JSON output functionality](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/json_integration_test.rs). Verifies that the CLI correctly integrates the JSON output formatter and produces expected JSON structure when the `--json` flag is used.
-
-11. **[tests/cli_normalization.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_normalization.rs)** (49 lines) - [Tests for CLI output normalization functionality](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_normalization.rs). Ensures cross-platform normalization helpers work correctly across different environments using snapshot testing.
+2. **[tests/cli_integration.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration.rs)** (887 lines) - [CLI integration tests using subprocess-based testing with assert_cmd](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration.rs). Provides natural process isolation and works correctly under llvm-cov for coverage reporting. Tests command-line interface functionality including multiple file processing, strict mode (`--strict`), built-in rules (`--use-builtin`), JSON output, stdin handling, error handling, timeout validation, shell completions, and custom magic files.
+
+3. **[tests/cli_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration_tests.rs)** (1,625 lines) - [CLI integration tests using the canonical libmagic test suite](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration_tests.rs). Verifies command-line interface functionality by running against third_party/tests/, covering canonical test suite compatibility and compatibility with original libmagic behavior.
+
+4. **[tests/property_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs)** (225 lines) - [Property-based tests verifying that evaluator never panics, buffer access is bounds-checked, metadata consistency, and serde roundtrips](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs). Includes [6 custom proptest strategies](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs#L18-L84): arb_offset_spec, arb_type_kind, arb_operator, arb_value, arb_magic_rule, arb_buffer.
+
+5. **[tests/evaluator_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/evaluator_tests.rs)** (248 lines) - [Evaluator integration tests through the public MagicDatabase API](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/evaluator_tests.rs). Tests confidence calculation, rule ordering, evaluation behavior, and validates that known file types return non-zero confidence scores.
+
+6. **[tests/compatibility_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility_tests.rs)** (360 lines) - [Compatibility tests ensuring identical results to the original libmagic](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility_tests.rs). [Test files are downloaded from the file/file repository](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility/README.md#L7) and compared against GNU file command.
+
+7. **[tests/parser_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/parser_integration_tests.rs)** (380 lines) - [End-to-end integration tests for magic file parser and database integration](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/parser_integration_tests.rs). Validates the complete flow from file/directory loading through rule evaluation.
+
+8. **[tests/directory_loading_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/directory_loading_tests.rs)** (431 lines) - [Integration tests for directory loading functionality](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/directory_loading_tests.rs). Validates `load_magic_directory()` function with various directory structures and batch loading of magic files from directories.
+
+9. **[tests/mime_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/mime_tests.rs)** (230 lines) - [MIME type mapping integration tests](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/mime_tests.rs). Tests MIME type detection through the public API including hardcoded mappings, prefix matching, and case insensitivity.
+
+10. **[tests/tags_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/tags_tests.rs)** (163 lines) - [Tag extraction integration tests](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/tags_tests.rs). Tests keyword extraction, case insensitivity, rule path tags, and custom keyword support.
+
+11. **[tests/json_integration_test.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/json_integration_test.rs)** (327 lines) - [Integration tests for JSON output functionality](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/json_integration_test.rs). Verifies that the CLI correctly integrates the JSON output formatter and produces expected JSON structure when the `--json` flag is used.
+
+12. **[tests/cli_normalization.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_normalization.rs)** (49 lines) - [Tests for CLI output normalization functionality](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_normalization.rs). Ensures cross-platform normalization helpers work correctly across different environments using snapshot testing.
 
 #### Supporting Test Files
 
@@ -75,6 +77,29 @@
 ```
 
 [The `common::normalize_cli_output()` function handles executable name conversion (rmagic.exe → rmagic), Windows path prefix removal (\\?\\), and error message filtering](https://app.dosu.dev/documents/691b7c0a-2533-474f-bdd5-11a429656050).
+
+### CLI Testing with assert_cmd
+
+The project uses subprocess-based CLI testing with assert_cmd for natural process isolation. [The tests/cli_integration.rs file](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration.rs) contains comprehensive CLI tests that run the rmagic binary as a subprocess:
+
+```rust
+use assert_cmd::Command;
+use predicates::prelude::*;
+
+#[test]
+fn test_builtin_elf_detection() {
+    let temp_dir = TempDir::new().expect("Failed to create temp dir");
+    let test_file = create_elf_file(&temp_dir);
+
+    Command::new(assert_cmd::cargo::cargo_bin!("rmagic"))
+        .args(["--use-builtin", test_file.to_str().expect("Invalid path")])
+        .assert()
+        .success()
+        .stdout(predicate::str::contains("ELF"));
+}
+```
+
+This approach provides reliable CLI testing that works correctly under llvm-cov for coverage reporting without platform-specific fd manipulation.
 
 ### Property-Based Testing with proptest
 
@@ -133,10 +158,12 @@
 ### Dev Dependencies
 
 [Core testing dependencies in Cargo.toml](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L162-L168):
+- **[assert_cmd 2.1.2](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L164)** - Subprocess-based CLI testing with process isolation
 - **[criterion 0.8.2](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L162)** - Benchmarking framework for performance measurement
 - **[insta 1.46.3](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L163)** - Snapshot testing library with JSON feature enabled
+- **[nix 0.31.2](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L164)** - Unix system API bindings with fs feature for filesystem testing
+- **[predicates 3.1.4](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L167)** - Assertion predicates for command output validation
 - **[proptest 1.10.0](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L166)** - Property-based testing framework
-- **[nix 0.31.2](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L164)** - Unix system API bindings with fs feature for filesystem testing
 - **[regex 1.12.3](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L165)** - Regular expression support for test assertions
 - **[tempfile 3.26.0](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/Cargo.toml#L167)** - Temporary file and directory creation for test fixtures
 
@@ -269,7 +296,8 @@
 | File Path | Description | Lines |
 |-----------|-------------|-------|
 | [tests/integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/integration_tests.rs) | Core end-to-end integration tests | 468 |
-| [tests/cli_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration_tests.rs) | Comprehensive CLI testing with canonical libmagic test suite | 1,625 |
+| [tests/cli_integration.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration.rs) | Subprocess-based CLI integration tests using assert_cmd | 887 |
+| [tests/cli_integration_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/cli_integration_tests.rs) | Canonical libmagic test suite compatibility tests | 1,625 |
 | [tests/property_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/property_tests.rs) | Property-based tests with proptest strategies | 225 |
 | [tests/evaluator_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/evaluator_tests.rs) | Evaluator integration tests through public API | 248 |
 | [tests/compatibility_tests.rs](https://github.com/EvilBit-Labs/libmagic-rs/blob/e8f5eb490731996b82b22ace849dd45832405cd1/tests/compatibility_tests.rs) | Compatibility validation with original libmagic | 360 |

✅ Accepted

testing-guidelines /libmagic-rs/blob/main/docs/src/testing-guidelines.md
View Suggested Changes
@@ -26,6 +26,7 @@
 │   └── evaluator/
 │       └── mod.rs          # Evaluator unit tests
 ├── tests/
+│   ├── cli_integration.rs  # CLI integration tests
 │   ├── integration/        # Integration tests
 │   ├── compatibility/      # GNU file compatibility tests
 │   └── fixtures/           # Test data and expected outputs
@@ -80,6 +81,47 @@
     assert_eq!(result.description, "ELF 64-bit LSB executable");
 }
 ```
+
+#### CLI Integration Tests
+
+Located in `tests/cli_integration.rs`, these tests verify the `rmagic` binary through subprocess execution using `assert_cmd` rather than testing internal functions. This approach provides proper process isolation and eliminates fragile file descriptor manipulation.
+
+Dependencies: `assert_cmd`, `predicates`, and `tempfile` (from dev-dependencies).
+
+```rust
+use assert_cmd::Command;
+use predicates::prelude::*;
+use tempfile::TempDir;
+
+#[test]
+fn test_builtin_elf_detection() {
+    let temp_dir = TempDir::new().expect("Failed to create temp dir");
+    let test_file = temp_dir.path().join("test.elf");
+    std::fs::write(&test_file, b"\x7fELF\x02\x01\x01\x00").unwrap();
+
+    Command::cargo_bin("rmagic")
+        .unwrap()
+        .args(["--use-builtin", test_file.to_str().unwrap()])
+        .assert()
+        .success()
+        .stdout(predicate::str::contains("ELF"));
+}
+```
+
+The test suite covers:
+
+- **Builtin Flag Tests**: Verify `--use-builtin` with various file formats (ELF, PNG, JPEG, PDF, ZIP, GIF)
+- **Stdin Tests**: Validate reading from stdin with `-`, including empty input and truncation warnings
+- **Multiple File Tests**: Sequential output, strict mode, JSON output, custom magic files
+- **Error Handling Tests**: Missing files, directories, invalid magic files, conflicting flags
+- **Timeout Tests**: Argument parsing, boundary conditions
+- **Output Format Tests**: Text and JSON formats for single and multiple files
+- **Shell Completion Tests**: Generate completion scripts for bash, zsh, fish
+- **Custom Magic File Tests**: User-provided magic file handling
+- **Edge Cases**: Files with spaces, Unicode names, empty files, small files
+- **CLI Argument Parsing Tests**: Multiple files, strict mode, format combinations
+
+Use CLI integration tests for end-to-end verification of `rmagic` binary behavior. Use unit tests (in `src/main.rs` or library modules) for testing individual functions and components in isolation.
 
 ## Writing Effective Tests
 
@@ -413,6 +455,12 @@
 cargo test ast_structures
 cargo test integration
 
+# Run CLI integration tests
+cargo test --test cli_integration
+
+# Run specific CLI test
+cargo test --test cli_integration test_builtin_elf_detection
+
 # Run tests with output
 cargo test -- --nocapture
 

✅ Accepted

Note: You must be authenticated to accept/decline updates.

How did I do? Any feedback?  Join Discord

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Mar 7, 2026

🧪 CI Insights

Here's what we observed from your CI run for cc00082.

🟢 All jobs passed!

But CI Insights is watching 👀

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extracts fragile CLI unit tests from src/main.rs into a dedicated tests/cli_integration.rs integration test file, replacing process-wide fd manipulation with assert_cmd-based subprocess testing. This addresses issue #61 which highlighted that main.rs was ~2.7x the 500-600 line guideline and used brittle dup/dup2 operations requiring a global mutex.

Changes:

  • New tests/cli_integration.rs (~887 lines) with subprocess-based CLI tests using assert_cmd, predicates, and tempfile
  • src/main.rs removes fragile fd-manipulation helpers (FD_MUTEX, capture_stdout, capture_stderr, with_mocked_stdin) and several migrated test functions
  • Cargo.toml adds assert_cmd, predicates as dev dependencies and bumps several dependency versions; adds unused user feature to the nix dev dependency
  • docs/src/testing.md updated to document the new integration test structure
  • mise.toml/mise.lock changes the Rust toolchain version from "stable" to "latest" (resolved to 1.93.1)
  • dist-workspace.toml purely cosmetic formatting changes

Reviewed changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/cli_integration.rs New file: 60 CLI integration tests using assert_cmd for subprocess-based testing
src/main.rs Removes fragile fd helpers, FD_MUTEX, and migrated test functions
Cargo.toml Adds assert_cmd, predicates; adds unused user feature to nix; bumps dependency versions
docs/src/testing.md Updates integration test documentation; lists only 2 of 11+ integration test files
mise.toml Changes Rust version from "stable" to "latest"
mise.lock Pins Rust to 1.93.1 (resolved from "latest")
dist-workspace.toml Cosmetic formatting only
Cargo.lock Lockfile updates for new/bumped dependencies

Comment thread Cargo.toml Outdated
Comment thread mise.toml
Comment thread docs/src/testing.md
Comment on lines +33 to +36
**Integration Tests**: Located in `tests/` directory:

- `tests/cli_integration.rs` - CLI subprocess tests using `assert_cmd`
- `tests/property_tests.rs` - Property-based tests using `proptest`
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Integration Tests" bullet list in docs/src/testing.md only mentions tests/cli_integration.rs and tests/property_tests.rs, but the tests/ directory contains many other integration test files: tests/cli_integration_tests.rs, tests/compatibility_tests.rs, tests/evaluator_tests.rs, tests/integration_tests.rs, tests/json_integration_test.rs, tests/mime_tests.rs, tests/parser_integration_tests.rs, tests/tags_tests.rs, and tests/directory_loading_tests.rs. The documentation gives the impression that only two integration test files exist, which is misleading.

Copilot uses AI. Check for mistakes.
Comment thread tests/cli_integration.rs
Comment on lines +454 to +463
rmagic_cmd()
.args([
"--magic-file",
"nonexistent.magic",
test_file.to_str().expect("Invalid path"),
])
.assert()
.failure()
.code(4)
.stderr(predicate::str::contains("Magic file"));
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test test_error_magic_file_not_found passes "nonexistent.magic" as a relative path to --magic-file. However, a file named nonexistent.magic actually exists in the repository root (it contains valid magic rules). Since integration test subprocesses typically inherit the workspace root as their working directory, this relative path will resolve to an existing valid magic file, causing the binary to succeed (exit code 0) instead of failing with exit code 4 as the test expects.

The fix is to use a path that is guaranteed not to exist, such as a path inside a TempDir that was not created (e.g., temp_dir.path().join("nonexistent.magic")), similar to how test_multiple_files_partial_failure_non_strict constructs temp_dir.path().join("nonexistent.bin").

Copilot uses AI. Check for mistakes.
unclesp1d3r and others added 2 commits March 6, 2026 19:22
…view findings

- Delete orphaned test infrastructure: cli_integration_tests.rs (1733 lines),
  cli_normalization.rs, common/mod.rs, and 6 stale snapshot files
- Consolidate 6 format-specific file creator helpers into constants + create_data_file
- Replace duplicate tests with table-driven patterns per AGENTS.md convention
- Strengthen PDF/GIF detection assertions (assert format name or "data")
- Replace from_utf8_lossy with from_utf8().expect() in JSON test
- Restore 7 missing unit-level argument parsing tests to main.rs
- Migrate pre-existing main.rs tests from manual cleanup to tempfile::TempDir
- Remove unused nix "user" feature from Cargo.toml
- Fix stale cli_integration_tests.rs references in builtin_rules.rs,
  Claude instincts/skills, plan docs, and testing.md
- Fix testing.md example to match actual rmagic_cmd() helper pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Copilot AI review requested due to automatic review settings March 7, 2026 00:24
@unclesp1d3r unclesp1d3r review requested due to automatic review settings March 7, 2026 00:24
- Fix test_error_magic_file_not_found using relative path that resolves
  to an existing file; use temp_dir path instead
- Expand integration test file list in testing.md to include all test
  files in tests/ directory

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Copilot AI review requested due to automatic review settings March 7, 2026 00:26
coderabbitai[bot]
coderabbitai Bot previously approved these changes Mar 7, 2026
@unclesp1d3r unclesp1d3r enabled auto-merge (squash) March 7, 2026 00:28
@unclesp1d3r unclesp1d3r added the lgtm Approved for the merge queue label Mar 7, 2026
@coderabbitai coderabbitai Bot added the documentation Improvements or additions to documentation label Mar 7, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 23 changed files in this pull request and generated 2 comments.

Comment thread Cargo.toml
[dev-dependencies]
assert_cmd = "2.1.2"
criterion = "0.8.2"
insta = { version = "1.46.3", features = ["json"] }
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The insta dev-dependency is still listed in Cargo.toml but is no longer used anywhere in the codebase. The tests/cli_integration_tests.rs and tests/cli_normalization.rs files that used assert_snapshot! have been deleted. The insta dependency should be removed from [dev-dependencies] to keep the build clean and avoid an unused dependency warning/audit finding.

Suggested change
insta = { version = "1.46.3", features = ["json"] }

Copilot uses AI. Check for mistakes.
Comment thread tests/cli_integration.rs
Comment on lines +145 to +157
let assertion = rmagic_cmd()
.args(["--use-builtin", "-"])
.write_stdin(*input)
.assert()
.success()
.stdout(predicate::str::contains("stdin:"));

if let Some(substr) = expected_substr {
assertion.stdout(predicate::str::contains(*substr));
}

// Satisfy the borrow checker - label is used for debugging context
let _ = label;
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The label variable in test_stdin_format_detection is bound in the destructuring pattern but immediately discarded with let _ = label; and a comment saying it is "used for debugging context". However, it is never actually passed to any assertion message. If a test case fails, there is no indication of which of the four cases triggered the failure, making debugging harder. The label should be included in the assertion message using .with_context(|| label.to_string()) or by refactoring into a helper that accepts a label and forwards it to the assertion, or alternatively by splitting this into separate named test functions.

Suggested change
let assertion = rmagic_cmd()
.args(["--use-builtin", "-"])
.write_stdin(*input)
.assert()
.success()
.stdout(predicate::str::contains("stdin:"));
if let Some(substr) = expected_substr {
assertion.stdout(predicate::str::contains(*substr));
}
// Satisfy the borrow checker - label is used for debugging context
let _ = label;
let output = rmagic_cmd()
.args(["--use-builtin", "-"])
.write_stdin(*input)
.output()
.expect("failed to run rmagic");
assert!(
output.status.success(),
"stdin case `{label}` failed: process exited with status {:?}",
output.status.code()
);
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(
stdout.contains("stdin:"),
"stdin case `{label}`: expected stdout to contain \"stdin:\", got: {stdout:?}"
);
if let Some(substr) = expected_substr {
assert!(
stdout.contains(substr),
"stdin case `{label}`: expected stdout to contain {substr:?}, got: {stdout:?}"
);
}

Copilot uses AI. Check for mistakes.
…support

find_rmagic_binary() only searched target/release/ and target/debug/, but
cargo llvm-cov builds to target/llvm-cov-target/. Use assert_cmd's
cargo_bin! macro as the primary lookup to find the binary regardless of
target directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
@unclesp1d3r unclesp1d3r merged commit 5f274da into main Mar 7, 2026
27 of 28 checks passed
@unclesp1d3r unclesp1d3r deleted the 61-refactor-extract-cli-tests-from-mainrs-into-integration-tests branch March 7, 2026 00:42
@github-actions github-actions Bot mentioned this pull request Mar 7, 2026
mergify Bot pushed a commit that referenced this pull request Mar 7, 2026
## 🤖 New release

* `libmagic-rs`: 0.4.2 -> 0.4.3 (✓ API compatible changes)

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

## [0.4.3] - 2026-03-07

### Refactor

- **tests**: Extract CLI tests from main.rs into integration tests
([#159](#159))
<!-- generated by git-cliff -->
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli Command-line interface and tools documentation Improvements or additions to documentation lgtm Approved for the merge queue size:XXL This PR changes 1000+ lines, ignoring generated files. testing Test infrastructure and coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: extract CLI tests from main.rs into integration tests

2 participants