Parser: implement comparison operators (<, >, <=, >=)

[unclesp1d3r]

Summary

Implement comparison operators <, >, <=, >= in both the parser and evaluator. These are currently explicitly rejected by the parser (grammar.rs) and are critical for compatibility with real magic files.

Context

Many magic rules use comparison operators for version checks, size validation, and range matching. Without these, the vast majority of system magic files cannot be fully interpreted.

Current state: Only =, !=, &, and &mask operators are implemented.

Acceptance Criteria

• Parser recognizes <, >, <=, >= operators in magic rule syntax
• Operator enum extended with LessThan, GreaterThan, LessEqual, GreaterEqual variants
• Evaluator applies comparison logic for all numeric types (byte, short, long)
• String comparison works lexicographically where applicable
• Unit tests for each operator with each type
• Integration tests with real magic rules using these operators
• All existing tests continue to pass

Impact

HIGH - Would unlock ~40% more magic file compatibility. Used extensively in format version detection, file size checks, and range-based matching.

Files to Modify

• src/parser/ast.rs - Extend Operator enum
• src/parser/grammar.rs - Parse new operators
• src/evaluator/operators.rs - Implement comparison logic
• Tests in all above modules

References

• GNU file magic(5) man page
• Epic Epic: libmagic-rs Phase 1 MVP #24

[traycerai]

Plan

Observations

The codebase currently supports only equality (=, ==), inequality (!=, <>), and bitwise AND (&, &mask) operators. The Operator enum in file:src/parser/ast.rs has 4 variants, the parser in file:src/parser/grammar.rs uses if-let chains to parse operators (lines 163-218), and the evaluator in file:src/evaluator/operators.rs dispatches to type-specific comparison functions. The existing implementation includes cross-type integer coercion for equality checks (Uint ↔ Int). Tests follow a comprehensive pattern covering all value types (Uint, Int, Bytes, String) with edge cases.

Approach

Extend the operator system by adding four new comparison variants to the Operator enum, updating the parser to recognize <, >, <=, >= symbols, and implementing comparison logic in the evaluator. The implementation will mirror the existing pattern: numeric comparisons for integer types with cross-type coercion, lexicographic comparison for strings, and false for incompatible type combinations. This approach maintains consistency with the existing codebase architecture while unlocking ~40% more magic file compatibility as stated in the ticket.

Implementation Steps

1. Extend the Operator Enum

File: file:src/parser/ast.rs

Add four new variants to the Operator enum (after line 114):

• LessThan - for < operator
• GreaterThan - for > operator
• LessEqual - for <= operator
• GreaterEqual - for >= operator

Update the enum documentation to describe each new variant with examples similar to existing operators.

2. Update the Parser

File: file:src/parser/grammar.rs

Modify the parse_operator function (lines 163-218) to recognize the new operators:

Add parsing logic for the new operators in order of precedence (longer operators first to avoid partial matches):

• Parse <= before < to avoid consuming just <
• Parse >= before > to avoid consuming just >
• Insert these checks after the existing != and <> checks but before single-character operators

Follow the existing pattern:

• Use tag() to match the operator string
• Check for invalid continuations (e.g., reject <<<)
• Consume trailing whitespace with multispace0
• Return the appropriate Operator variant

Add unit tests in the test module (around line 500+) following the existing pattern:

• Test each operator individually
• Test with surrounding whitespace
• Test rejection of invalid sequences
• Test in context of complete magic rules

3. Implement Comparison Logic in Evaluator

File: file:src/evaluator/operators.rs

Create four new comparison functions following the pattern of apply_equal (lines 9-66):

Function: apply_less_than

• For Uint vs Uint: direct comparison
• For Int vs Int: direct comparison
• For Uint vs Int: convert to common type if safe, compare
• For Int vs Uint: convert to common type if safe, compare
• For String vs String: lexicographic comparison using <
• For Bytes vs Bytes: lexicographic comparison
• For incompatible types: return false

Function: apply_greater_than

• Mirror apply_less_than logic with reversed comparison

Function: apply_less_equal

• Combine apply_less_than and apply_equal logic using ||

Function: apply_greater_equal

• Combine apply_greater_than and apply_equal logic using ||

Update apply_operator function (lines 217-237) to dispatch to new functions:

• Add match arms for Operator::LessThan, Operator::GreaterThan, Operator::LessEqual, Operator::GreaterEqual
• Call corresponding comparison functions

4. Add Comprehensive Unit Tests

File: file:src/evaluator/operators.rs (test module starting line 239)

For each new operator, add test cases following the existing pattern:

Basic numeric tests (similar to lines 243-308):

• Same values (should match for <=, >=, not for <, >)
• Different values in both directions
• Zero values
• Extreme values (MAX, MIN)

Cross-type integer tests (similar to lines 401-420):

• Uint vs Int with same numeric value
• Positive vs negative
• Large values that don't fit in opposite type

String comparison tests (similar to lines 350-397):

• Lexicographic ordering ("a" < "b")
• Same strings
• Empty strings
• Case sensitivity
• Unicode strings

Bytes comparison tests (similar to lines 311-347):

• Lexicographic byte ordering
• Different lengths
• Empty byte sequences

Incompatible type tests (similar to lines 423-455):

• Numeric vs String (should return false)
• Numeric vs Bytes (should return false)
• String vs Bytes (should return false)

Edge cases:

• Boundary values for <= and >=
• Transitivity tests (if a < b and b < c, then a < c)

5. Add Parser Integration Tests

File: file:tests/parser_integration_tests.rs

Add tests for parsing magic rules with comparison operators:

• Parse rules with < operator: 0 long <100 small value
• Parse rules with > operator: 0 long >1000 large value
• Parse rules with <= operator: 0 short <=255 byte range
• Parse rules with >= operator: 0 byte >=0x80 high bit set
• Test with different numeric types (byte, short, long)
• Test with string values for lexicographic comparison
• Test nested rules using comparison operators

6. Add Evaluator Integration Tests

File: file:tests/evaluator_tests.rs

Add integration tests using real magic rule patterns:

Version detection pattern:

0 long >0x01000000 version 1.x or higher

Size validation pattern:

4 leshort <1024 small file

Range checking pattern:

0 byte >=0x20 printable ASCII range
0 byte <=0x7E printable ASCII range

Test with actual file buffers that should match/not match these patterns.

7. Update Documentation

File: file:docs/MAGIC_FORMAT.md

Update the "Operators" section (lines 187-220):

• Move >, < from examples to "Currently Supported" list
• Add >=, <= to the operator table with descriptions
• Add examples showing usage with different types
• Update "Not Yet Supported" section (line 477) to remove comparison operators

Update "Supported Features" section (lines 463-488):

• Add "Comparison operators (<, >, <=, >=)" to "Currently Supported"

8. Verify All Tests Pass

Run the full test suite to ensure:

• All existing tests continue to pass (regression check)
• New operator tests pass
• Parser correctly rejects invalid operator sequences
• Evaluator handles all type combinations correctly
• Integration tests with real magic patterns work

sequenceDiagram
    participant Parser as Parser<br/>(grammar.rs)
    participant AST as AST<br/>(ast.rs)
    participant Evaluator as Evaluator<br/>(operators.rs)
    participant Tests as Test Suite

    Note over Parser,Evaluator: Step 1: Extend AST
    AST->>AST: Add LessThan, GreaterThan,<br/>LessEqual, GreaterEqual variants

    Note over Parser,Evaluator: Step 2: Update Parser
    Parser->>Parser: Add <= and >= parsing<br/>(before < and >)
    Parser->>Parser: Add < and > parsing
    Parser->>AST: Return new Operator variants

    Note over Parser,Evaluator: Step 3: Implement Evaluator
    Evaluator->>Evaluator: Create apply_less_than()
    Evaluator->>Evaluator: Create apply_greater_than()
    Evaluator->>Evaluator: Create apply_less_equal()
    Evaluator->>Evaluator: Create apply_greater_equal()
    Evaluator->>Evaluator: Update apply_operator()<br/>dispatch logic

    Note over Parser,Evaluator: Step 4-6: Testing
    Tests->>Parser: Test operator parsing
    Tests->>Evaluator: Test comparison logic<br/>(all type combinations)
    Tests->>Evaluator: Test integration with<br/>real magic patterns

    Note over Parser,Evaluator: Step 7: Documentation
    Tests->>AST: Verify all tests pass

Loading

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase currently supports only equality (`=`, `==`), inequality (`!=`, `<>`), and bitwise AND (`&`, `&mask`) operators. The `Operator` enum in `file:src/parser/ast.rs` has 4 variants, the parser in `file:src/parser/grammar.rs` uses if-let chains to parse operators (lines 163-218), and the evaluator in `file:src/evaluator/operators.rs` dispatches to type-specific comparison functions. The existing implementation includes cross-type integer coercion for equality checks (Uint ↔ Int). Tests follow a comprehensive pattern covering all value types (Uint, Int, Bytes, String) with edge cases.

## Approach

Extend the operator system by adding four new comparison variants to the `Operator` enum, updating the parser to recognize `<`, `>`, `<=`, `>=` symbols, and implementing comparison logic in the evaluator. The implementation will mirror the existing pattern: numeric comparisons for integer types with cross-type coercion, lexicographic comparison for strings, and false for incompatible type combinations. This approach maintains consistency with the existing codebase architecture while unlocking ~40% more magic file compatibility as stated in the ticket.

## Implementation Steps

### 1. Extend the Operator Enum

**File**: `file:src/parser/ast.rs`

Add four new variants to the `Operator` enum (after line 114):
- `LessThan` - for `<` operator
- `GreaterThan` - for `>` operator  
- `LessEqual` - for `<=` operator
- `GreaterEqual` - for `>=` operator

Update the enum documentation to describe each new variant with examples similar to existing operators.

### 2. Update the Parser

**File**: `file:src/parser/grammar.rs`

Modify the `parse_operator` function (lines 163-218) to recognize the new operators:

Add parsing logic for the new operators in order of precedence (longer operators first to avoid partial matches):
- Parse `<=` before `<` to avoid consuming just `<`
- Parse `>=` before `>` to avoid consuming just `>`
- Insert these checks after the existing `!=` and `<>` checks but before single-character operators

Follow the existing pattern:
- Use `tag()` to match the operator string
- Check for invalid continuations (e.g., reject `<<<`)
- Consume trailing whitespace with `multispace0`
- Return the appropriate `Operator` variant

Add unit tests in the test module (around line 500+) following the existing pattern:
- Test each operator individually
- Test with surrounding whitespace
- Test rejection of invalid sequences
- Test in context of complete magic rules

### 3. Implement Comparison Logic in Evaluator

**File**: `file:src/evaluator/operators.rs`

Create four new comparison functions following the pattern of `apply_equal` (lines 9-66):

**Function: `apply_less_than`**
- For `Uint` vs `Uint`: direct comparison
- For `Int` vs `Int`: direct comparison  
- For `Uint` vs `Int`: convert to common type if safe, compare
- For `Int` vs `Uint`: convert to common type if safe, compare
- For `String` vs `String`: lexicographic comparison using `<`
- For `Bytes` vs `Bytes`: lexicographic comparison
- For incompatible types: return `false`

**Function: `apply_greater_than`**
- Mirror `apply_less_than` logic with reversed comparison

**Function: `apply_less_equal`**
- Combine `apply_less_than` and `apply_equal` logic using `||`

**Function: `apply_greater_equal`**
- Combine `apply_greater_than` and `apply_equal` logic using `||`

Update `apply_operator` function (lines 217-237) to dispatch to new functions:
- Add match arms for `Operator::LessThan`, `Operator::GreaterThan`, `Operator::LessEqual`, `Operator::GreaterEqual`
- Call corresponding comparison functions

### 4. Add Comprehensive Unit Tests

**File**: `file:src/evaluator/operators.rs` (test module starting line 239)

For each new operator, add test cases following the existing pattern:

**Basic numeric tests** (similar to lines 243-308):
- Same values (should match for `<=`, `>=`, not for `<`, `>`)
- Different values in both directions
- Zero values
- Extreme values (MAX, MIN)

**Cross-type integer tests** (similar to lines 401-420):
- `Uint` vs `Int` with same numeric value
- Positive vs negative
- Large values that don't fit in opposite type

**String comparison tests** (similar to lines 350-397):
- Lexicographic ordering ("a" < "b")
- Same strings
- Empty strings
- Case sensitivity
- Unicode strings

**Bytes comparison tests** (similar to lines 311-347):
- Lexicographic byte ordering
- Different lengths
- Empty byte sequences

**Incompatible type tests** (similar to lines 423-455):
- Numeric vs String (should return false)
- Numeric vs Bytes (should return false)
- String vs Bytes (should return false)

**Edge cases**:
- Boundary values for `<=` and `>=`
- Transitivity tests (if a < b and b < c, then a < c)

### 5. Add Parser Integration Tests

**File**: `file:tests/parser_integration_tests.rs`

Add tests for parsing magic rules with comparison operators:
- Parse rules with `<` operator: `0 long <100 small value`
- Parse rules with `>` operator: `0 long >1000 large value`  
- Parse rules with `<=` operator: `0 short <=255 byte range`
- Parse rules with `>=` operator: `0 byte >=0x80 high bit set`
- Test with different numeric types (byte, short, long)
- Test with string values for lexicographic comparison
- Test nested rules using comparison operators

### 6. Add Evaluator Integration Tests

**File**: `file:tests/evaluator_tests.rs`

Add integration tests using real magic rule patterns:

**Version detection pattern**:
```
0 long >0x01000000 version 1.x or higher
```

**Size validation pattern**:
```  
4 leshort <1024 small file
```

**Range checking pattern**:
```
0 byte >=0x20 printable ASCII range
0 byte <=0x7E printable ASCII range
```

Test with actual file buffers that should match/not match these patterns.

### 7. Update Documentation

**File**: `file:docs/MAGIC_FORMAT.md`

Update the "Operators" section (lines 187-220):
- Move `>`, `<` from examples to "Currently Supported" list
- Add `>=`, `<=` to the operator table with descriptions
- Add examples showing usage with different types
- Update "Not Yet Supported" section (line 477) to remove comparison operators

Update "Supported Features" section (lines 463-488):
- Add "Comparison operators (<, >, <=, >=)" to "Currently Supported"

### 8. Verify All Tests Pass

Run the full test suite to ensure:
- All existing tests continue to pass (regression check)
- New operator tests pass
- Parser correctly rejects invalid operator sequences
- Evaluator handles all type combinations correctly
- Integration tests with real magic patterns work

```mermaid
sequenceDiagram
    participant Parser as Parser<br/>(grammar.rs)
    participant AST as AST<br/>(ast.rs)
    participant Evaluator as Evaluator<br/>(operators.rs)
    participant Tests as Test Suite

    Note over Parser,Evaluator: Step 1: Extend AST
    AST->>AST: Add LessThan, GreaterThan,<br/>LessEqual, GreaterEqual variants

    Note over Parser,Evaluator: Step 2: Update Parser
    Parser->>Parser: Add <= and >= parsing<br/>(before < and >)
    Parser->>Parser: Add < and > parsing
    Parser->>AST: Return new Operator variants

    Note over Parser,Evaluator: Step 3: Implement Evaluator
    Evaluator->>Evaluator: Create apply_less_than()
    Evaluator->>Evaluator: Create apply_greater_than()
    Evaluator->>Evaluator: Create apply_less_equal()
    Evaluator->>Evaluator: Create apply_greater_equal()
    Evaluator->>Evaluator: Update apply_operator()<br/>dispatch logic

    Note over Parser,Evaluator: Step 4-6: Testing
    Tests->>Parser: Test operator parsing
    Tests->>Evaluator: Test comparison logic<br/>(all type combinations)
    Tests->>Evaluator: Test integration with<br/>real magic patterns

    Note over Parser,Evaluator: Step 7: Documentation
    Tests->>AST: Verify all tests pass
```

---

Execution Information

Branch: main
Commit: 36adac5

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

Warning

Something went wrong while processing your request.

Reference ID: c92576cd-fbfe-4dd6-8e84-8241be352dfd

Please try again or contact support if the issue persists, referencing this ID.