fix(tests): Handle Windows drive letters in CLI output normalization#10
Merged
Conversation
Merged
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Comment |
Fixed the CLI integration test's normalize_cli_output function to correctly handle Windows paths that contain colons (e.g., D:\path\to\file). The previous implementation would incorrectly find the first colon (from the drive letter) instead of the colon separating the filename from the description. The fix searches for the filename followed by ": " pattern, ensuring we find the correct delimiter regardless of drive letters or other colons in the path. This resolves the test-cross-platform Windows CI failures where the output was showing "CVE-2014-1943.testfile: data" instead of just "data". Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com>
Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add compatibility testing infrastructure and update CI/CD workflows
fix(tests): Handle Windows drive letters in CLI output normalization
Dec 15, 2025
unclesp1d3r
added a commit
that referenced
this pull request
Dec 15, 2025
* feat(cli): Implement robust CLI argument parsing and file type detection - Add comprehensive CLI argument structure using clap derive macros - Implement platform-specific magic file path resolution - Add support for JSON and text output formats - Create `Args` struct with flexible configuration options - Add default magic file path detection for Unix and Windows - Implement output format selection with `OutputFormat` enum - Add basic error handling and file existence validation - Include initial test coverage for CLI argument parsing Enhances the CLI tool's usability and provides a flexible interface for file type identification across different platforms. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(tests): Add compatibility testing framework and CI integration - Introduced a new compatibility testing suite to ensure that libmagic-rs produces identical results to the original libmagic implementation. - Added a `tests/compatibility_tests.rs` file to run tests against the original test files from the file/file repository. - Created a `justfile` with commands for initializing and running compatibility tests, including CI integration for automated testing on push and pull request events. - Implemented a GitHub Actions workflow in `compatibility.yml` to run compatibility tests across multiple platforms, ensuring consistent results. - Added a `.gitmodules` file to manage the file-tests submodule for compatibility tests. These enhancements improve the testing infrastructure, ensuring robust validation of the library's functionality against established benchmarks. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(magic): Add basic magic file for testing and enhance CLI functionality - Introduced a basic magic file (`missing.magic` and `nonexistent.magic`) for testing and CI/CD environments, containing signatures for various file types including ELF, PE, ZIP, JPEG, PNG, GIF, PDF, and common text patterns. - Updated the `justfile` to include specific actionlint commands for various GitHub workflows, ensuring comprehensive linting coverage. - Enhanced the CLI in `main.rs` to check for the existence of the magic file and attempt to download it if missing, providing a fallback mechanism for CI/CD environments. - Implemented a new function `download_magic_files` to create a basic magic file when none exists, improving usability in testing scenarios. These changes improve the testing infrastructure and ensure robust functionality for file type detection in various environments. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(cli): Enhance error handling and validation in CLI - Implemented comprehensive error handling in the CLI, providing user-friendly messages for various error scenarios, including file not found, permission denied, and invalid input. - Added validation functions to ensure command-line arguments are correctly formatted and accessible before processing. - Updated the `run_analysis` function to utilize the new error handling mechanism, improving the robustness of the CLI tool. - Introduced new tests to validate error handling and argument validation, ensuring consistent behavior across different input scenarios. These enhancements improve the user experience by providing clearer feedback and ensuring that the CLI operates reliably under various conditions. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * chore(third_party): Update third-party test files and configurations - Added NOTICE and COPYING from `github.com/file/file` to provide proper attribution for test files. - Refactored various third-party test files and configurations to ensure consistency and maintainability. - Updated `.gitattributes`, `.mdformat.toml`, and `.prettierignore` for prevent alterations to the test files. - Cleaned up and organized test results and test files across multiple formats to enhance clarity and usability. These changes contribute to a more organized testing framework and improve the overall structure of third-party resources. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(third_party): Add magic.mgc file for file type detection testing - Introduced a new binary file `magic.mgc` in the `third_party` directory to support file type detection testing. - This addition enhances the testing framework by providing a standard magic file for validating the functionality of the library. These changes contribute to a more robust testing environment and ensure accurate file type detection capabilities. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * chore(tests): Remove submodule for compatibility tests and update verification process - Deleted the `.gitmodules` entry for the `file-tests` submodule, streamlining the project structure. - Updated the `justfile` to replace the submodule initialization command with a verification command that checks for the existence of compatibility test files in the `third_party` directory. - Adjusted the GitHub Actions workflow to reflect the new verification process for compatibility test files, ensuring that the necessary files are available before proceeding with builds. These changes simplify the setup process for compatibility tests and enhance the reliability of the testing framework. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(output): Add JSON output support for file type detection - Implement JSON output formatting with JsonMatchResult struct - Add JSON output integration in main.rs with --json flag handling - Create src/output/json.rs to define JSON serialization structure - Update tasks.md to mark JSON output implementation tasks as completed - Add tempfile dependency in Cargo.toml for testing - Convert evaluation results to structured JSON match results - Enhance CLI output routing to support JSON format - Provide machine-readable file type detection results Closes #11 (JSON match result implementation) Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * refactor(validation): Improve configuration validation with modular checks - Split `validate()` method into separate validation functions - Add constants for safe thresholds in recursion depth, string length, and timeout - Enhance error messages with more descriptive and context-specific details - Separate concerns by creating individual validation methods for different aspects - Improve error handling and prevent potential resource exhaustion scenarios - Maintain consistent error reporting and validation logic This refactoring improves the robustness and readability of configuration validation, making it easier to understand and maintain security checks. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * refactor(error): Improve error handling and message formatting in CLI and library - Refactor error handling functions to use more concise, multi-line error messages - Optimize error message generation with reduced string allocations - Update error handling in main.rs to consolidate error printing logic - Modify create_basic_magic_content() to use a const for magic file content - Improve JSON output generation by using pre-allocated vector - Standardize error message formatting across different error types - Reduce redundant string formatting and improve readability of error messages These changes enhance the error reporting mechanism, making error messages more consistent and slightly more memory-efficient while maintaining clear and informative output. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(evaluator): Add support for string type evaluation in magic rules - Implement string type reading in evaluator with null-termination support - Update test cases to validate string type evaluation - Remove unsupported string type error handling - Add test for matching and non-matching string rules - Mark string type implementation tasks as completed in tasks.md Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(magic): Add basic string type support in AST and matching - Implement basic string type in AST - Add string matching support with UTF-8 validation - Extend read_typed_value function to handle String type - Mark tasks 12, 12.1, and 12.3 as completed in implementation plan - Prepare for advanced string parsing and validation Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * refactor(validation): Improve error handling and message formatting in configuration validation - Update error messages to use dynamic formatting with constants - Add more descriptive error messages for configuration validation - Enhance error context in validation methods for recursion depth, string length, and timeout - Improve test assertions to check for more specific error message details - Remove TODO comments and replace with actual implementation improvements - Add runtime context to error messages using predefined constants Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * refactor(performance): Optimize JSON and parsing operations with iterator methods - Replace manual vector building with iterator methods in JSON output functions - Use `with_capacity()` to pre-allocate vectors for better performance - Improve hex byte parsing in grammar parser with direct digit conversion - Remove unnecessary `format!()` allocations in hex parsing - Reduce memory allocations and improve parsing efficiency These changes focus on reducing memory allocations and improving performance in key parsing and output generation functions. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * chore(hooks): Simplify CI auto-fix hook configuration - Remove unnecessary file patterns from hook trigger configuration - Reduce scope of hook to only watch Rust source files - Trim trailing newline for consistency - Maintain core auto-fix agent prompt and functionality Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(error): Add comprehensive error handling for libmagic-rs - Create src/error.rs with detailed error types using thiserror - Implement LibmagicError enum with variants for parsing, evaluation, and I/O errors - Add detailed ParseError enum with specific error types for magic file parsing - Create EvaluationError enum to handle rule evaluation and type reading errors - Implement helper methods for creating specific error instances - Mark task 13.1 as completed in tasks.md - Provide comprehensive error reporting with context and line numbers Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(error): Add evaluation error types for runtime detection - Create EvaluationError enum in error.rs module - Add variants for BufferOverrun, InvalidOffset, and UnsupportedType - Mark task 13.2 as completed in implementation plan - Improve error handling for runtime evaluation scenarios Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(evaluator): Implement graceful error handling in rule evaluation - Add comprehensive error handling for individual rule evaluation - Implement graceful degradation to skip problematic rules - Log warning messages for skipped rules with specific error details - Enhance error context in evaluate_rules and evaluate_single_rule functions - Preserve evaluation flow by continuing processing after encountering rule-level errors - Maintain critical error handling for timeout and recursion limit scenarios - Update documentation to reflect new error handling approach Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * chore(ci): Update CI configuration for libmagic-rs - Change Codecov slug to match the repository name - Update Rust toolchain version to 1.90 - Remove caching of cargo dependencies and replace with installation of the Just task runner This update streamlines the CI process and ensures compatibility with the latest Rust toolchain. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * chore(dependencies): Update dependencies and improve test handling - Add `regex` and `temp-env` dependencies for enhanced functionality. - Update `read_long` documentation to clarify behavior regarding NUL characters. - Introduce `BuildNoiseFilter` struct to filter out build noise in CLI integration tests. - Refactor test file creation to use `NamedTempFile` for automatic cleanup. - Normalize output in CLI integration tests to improve snapshot consistency. These changes enhance the testing framework and clarify the behavior of string reading functions, contributing to overall code quality and maintainability. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * test(cli): normalize rmagic(.exe) via insta filters for cross-platform snapshots - Add normalize_cli_output() helper in tests/common/mod.rs to handle: - Convert 'rmagic.exe' to 'rmagic' for Windows compatibility - Remove Windows path prefixes (\\?\) - Filter cargo-specific error messages - Update CLI integration tests to use normalization helper - Add comprehensive regression tests in tests/cli_normalization.rs - Update all affected insta snapshots to use normalized output Fixes cross-platform test failures where Windows .exe suffix caused snapshot mismatches between Unix and Windows environments. * build(justfile): improve cross-platform coverage commands - Split coverage and coverage-check targets into [unix] and [windows] variants - Add proper environment setup with RUSTFLAGS for coverage builds - Clean coverage artifacts before generating reports on both platforms - Use bash script with set -euo pipefail for Unix reliability - Use PowerShell syntax for Windows commands Ensures consistent coverage generation across different operating systems and improves reliability of coverage reports in CI environments. * refactor(project-structure): Reorganize test files and update documentation paths - Remove `test_files/` directory from project - Update documentation references to use `third_party/` for test files and magic files - Modify code examples and test paths to reflect new directory structure - Update development, getting-started, and testing documentation - Adjust `.gitignore` to remove `test_files/` entry - Update task tracking to reflect ongoing implementation status Rationale: Improve project organization and standardize test file management by using a more descriptive `third_party/` directory for external test resources and magic files. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(tests): normalize file paths in CLI integration test snapshots - Add path normalization utilities in tests/common/mod.rs - Normalize absolute paths to relative filenames in CLI test outputs - Update CLI integration test to use path normalization before snapshot assertion - Add unit tests for path normalization functions - Accept updated snapshot with normalized paths for cross-platform compatibility This ensures snapshot tests work consistently across different development environments by removing system-specific absolute paths from test outputs. * chore(tasks): Update implementation progress tracking in project tasks - Mark tasks 11-13 as completed in implementation plan - Update task status for JSON match result structure - Update task status for basic string type in AST - Update task status for basic error types - Add new task for text-based magic file parsing - Add detailed notes and requirements for upcoming implementation tasks - Prepare project roadmap for next development phases Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * chore(tasks): Update implementation progress tracking in project tasks - Completed comprehensive project implementation milestones - Updated task tracking with detailed progress descriptions - Documented completed stages for Rust libmagic implementation - Included summaries for key development areas: * Project structure and dependencies * AST type definitions * Parser components * Memory-mapped file I/O * Offset resolution * Type reading and interpretation * Operator evaluation * Rule evaluation engine * Output formatting system - Provides clear overview of project development status and achievements Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * feat(parser): Enhance magic rule parsing with advanced value and operator support - Add BitwiseAndMask variant to Operator enum for masked bitwise comparisons - Implement bitwise masking logic in apply_operator function - Extend hex byte parsing to support mixed hex and ASCII sequences - Update test coverage for new bitwise masking operator - Mark text format magic rule parsing task as complete in implementation tasks - Improve parsing flexibility for complex magic file rule formats Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> * fix: resolve clippy warning in test_all_offset_spec_variants (#8) * Initial plan * fix: replace vec! with array in test_all_offset_spec_variants Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com> * fix(tests): Handle Windows drive letters in CLI output normalization (#10) * Initial plan * fix(tests): Handle Windows drive letters in path normalization Fixed the CLI integration test's normalize_cli_output function to correctly handle Windows paths that contain colons (e.g., D:\path\to\file). The previous implementation would incorrectly find the first colon (from the drive letter) instead of the colon separating the filename from the description. The fix searches for the filename followed by ": " pattern, ensuring we find the correct delimiter regardless of drive letters or other colons in the path. This resolves the test-cross-platform Windows CI failures where the output was showing "CVE-2014-1943.testfile: data" instead of just "data". Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com> * chore: Remove accidentally committed test file Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: unclesp1d3r <251112+unclesp1d3r@users.noreply.github.com> --------- Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Windows CI test failures were caused by
normalize_cli_outputintests/cli_integration_tests.rsincorrectly parsing paths with drive letters. The function used.find(':')to locate the filename-description separator, which matched the drive letter colon (e.g.,D:) instead of the delimiter after the filename.Changes
normalize_cli_outputto search for the complete pattern"{filename}: "instead of finding the first colonExample
This fixes all 81 test failures in the
test-cross-platformWindows CI job.💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.