Parser: implement quad (64-bit integer) type with endian variants

[unclesp1d3r]

Summary

Implement the quad type (64-bit integer) and its endianness variants: quad, lequad, bequad.

Context

64-bit integer fields are essential for modern file formats. Many binary formats store sizes, offsets, and identifiers as 64-bit values.

Current state: Only byte (8-bit), short (16-bit), long (32-bit), and string types are implemented.

Acceptance Criteria

• TypeKind enum extended with Quad variant
• Parser recognizes quad, lequad, bequad type specifiers
• read_quad() function in evaluator/types.rs reads 8 bytes with correct endianness
• All operators work with quad values (=, !=, &, and future comparison operators)
• Overflow-safe arithmetic for 64-bit values
• Unit tests for each endian variant
• Integration tests with real magic rules

Impact

HIGH - Required for modern binary formats, large file handling, and 64-bit timestamp fields.

Files to Modify

• src/parser/ast.rs - Extend TypeKind enum
• src/parser/grammar.rs - Parse quad type specifiers
• src/evaluator/types.rs - Implement read_quad()
• src/evaluator/operators.rs - Ensure operator support for u64/i64

References

• GNU file magic(5) man page
• Epic Epic: libmagic-rs Phase 1 MVP #24

[traycerai]

Plan

Observations

The codebase follows a consistent pattern for implementing integer types: Byte (8-bit), Short (16-bit), and Long (32-bit) are already implemented with endianness variants. The TypeKind enum in file:src/parser/ast.rs uses structured variants with endian and signed fields for multi-byte types. The byteorder crate provides read_u64 methods for 64-bit reading. The Value enum already supports u64 and i64 through Value::Uint and Value::Int, and operators in file:src/evaluator/operators.rs already handle these types correctly.

Approach

Following the established pattern for Short and Long types, implement the Quad variant as a structured enum with endianness and signedness support. The implementation will mirror the existing 16-bit and 32-bit implementations, using the byteorder crate's read_u64 method for reading 8 bytes. The parser will recognize quad, lequad, and bequad type specifiers, mapping them to the appropriate TypeKind::Quad variant with corresponding endianness. This approach ensures consistency with the existing codebase and maintains compatibility with GNU file's magic format.

Implementation Steps

1. Extend TypeKind Enum in AST

File: file:src/parser/ast.rs

Add the Quad variant to the TypeKind enum after the Long variant (around line 96):

/// 64-bit integer
Quad {
    /// Byte order
    endian: Endianness,
    /// Whether value is signed
    signed: bool,
},

Update the test section to include tests for the new Quad variant, following the pattern of existing test_type_kind_short and test_type_kind_long tests (around line 520-550).

2. Update Grammar Parser

File: file:src/parser/grammar.rs

In the parse_type_and_operator function (around line 1432), add quad type specifiers to the alt() combinator before the existing types to ensure longest-match-first parsing:

let (input, type_name) = alt((
    tag("lequad"),   // Add first
    tag("bequad"),   // Add second
    tag("quad"),     // Add third
    tag("lelong"),
    tag("belong"),
    // ... rest of existing types
))

In the match statement (around line 1458), add mappings for the quad types:

"quad" => TypeKind::Quad {
    endian: Endianness::Native,
    signed: false,
},
"lequad" => TypeKind::Quad {
    endian: Endianness::Little,
    signed: false,
},
"bequad" => TypeKind::Quad {
    endian: Endianness::Big,
    signed: false,
},

Add comprehensive tests in the test section (around line 1809-1879) following the pattern of test_parse_type_endianness:

• Test parsing quad, lequad, bequad
• Test with whitespace
• Test with remaining input
• Test in complete magic rules

3. Implement read_quad Function

File: file:src/evaluator/types.rs

Add the read_quad function after read_long (around line 207), following the exact pattern of read_short and read_long:

Function signature:

pub fn read_quad(
    buffer: &[u8],
    offset: usize,
    endian: Endianness,
    signed: bool,
) -> Result<Value, TypeReadError>

Implementation details:

• Extract 8 bytes using buffer.get(offset..offset + 8)
• Return TypeReadError::BufferOverrun if insufficient bytes
• Use LittleEndian::read_u64, BigEndian::read_u64, or NativeEndian::read_u64 based on endianness
• For signed values, cast u64 to i64 and return Value::Int(value as i64) with #[allow(clippy::cast_possible_wrap)]
• For unsigned values, return Value::Uint(value)

Documentation: Include comprehensive doc comments with examples showing:

• Reading unsigned little-endian quad
• Reading signed values
• Buffer overrun error cases

4. Update read_typed_value Function

File: file:src/evaluator/types.rs

In the read_typed_value function (around line 307-358), add a new match arm for TypeKind::Quad:

TypeKind::Quad { endian, signed } => read_quad(buffer, offset, *endian, *signed),

This should be added after the TypeKind::Long arm and before TypeKind::String.

5. Verify Operator Support

File: file:src/evaluator/operators.rs

Review the operator implementations to confirm they already handle u64 and i64 values correctly:

• apply_equal (lines 9-66): Already handles Value::Uint and Value::Int with cross-type coercion
• apply_not_equal (lines 68-107): Delegates to apply_equal
• apply_bitwise_and (lines 109-163): Already handles bitwise operations on u64 and i64
• apply_operator (lines 165-237): Handles BitwiseAndMask with u64 values

No changes required - operators already support 64-bit values through the existing Value::Uint(u64) and Value::Int(i64) variants.

6. Add Unit Tests

File: file:src/evaluator/types.rs

Add comprehensive unit tests in the test module (after line 360), following the pattern of read_short and read_long tests:

Test coverage:

• test_read_quad_little_endian_unsigned: Test reading 0x0123456789ABCDEF in little-endian
• test_read_quad_big_endian_unsigned: Test reading in big-endian
• test_read_quad_native_endian_unsigned: Test native endianness
• test_read_quad_little_endian_signed: Test signed interpretation with positive values
• test_read_quad_signed_negative: Test negative values (e.g., 0xFFFFFFFFFFFFFFFF as -1)
• test_read_quad_buffer_overrun: Test error when buffer has fewer than 8 bytes
• test_read_quad_at_buffer_boundary: Test reading at exact buffer end
• test_read_quad_max_values: Test u64::MAX and i64::MAX/i64::MIN
• test_read_quad_zero: Test reading zero value

File: file:src/parser/grammar.rs

Add parser tests in the test section (around line 1880):

• test_parse_quad_types: Test parsing all three quad variants
• test_parse_quad_with_operator: Test lequad&0xFFFFFFFF00000000
• test_parse_magic_rule_with_quad: Test complete magic rules using quad types

7. Add Integration Tests

File: file:tests/integration_tests.rs

Add integration tests demonstrating real-world usage:

Test scenarios:

• Create a magic rule that detects 64-bit file signatures (e.g., ELF64 class field)
• Test reading 64-bit timestamps or file sizes
• Test with actual binary data containing 64-bit values
• Test endianness variants with known binary formats

Example test structure:

#[test]
fn test_quad_type_detection() {
    // Create magic file with quad rule
    // Test with buffer containing 64-bit value
    // Verify correct detection and value interpretation
}

8. Overflow Safety Verification

Files: file:src/evaluator/types.rs, file:src/evaluator/operators.rs

Verify overflow-safe arithmetic:

• The read_quad function uses direct casting from u64 to i64 for signed interpretation, which is safe as it preserves bit patterns
• Operators use bitwise operations on u64 values, which don't overflow
• Comparison operators use type-safe comparisons without arithmetic
• The Value enum already stores u64 and i64 directly, avoiding intermediate conversions

No additional overflow protection needed - the implementation follows Rust's safe casting patterns and the existing codebase's approach.

9. Documentation Updates

File: file:src/parser/grammar.rs

Update the doc comment for parse_type_and_operator (around line 1402-1423) to include quad types:

/// - `quad` - 64-bit integer (native endian)
/// - `lequad` - 64-bit little-endian integer
/// - `bequad` - 64-bit big-endian integer

File: file:src/evaluator/types.rs

Update the module-level documentation (lines 1-5) to mention 64-bit integer support.

Testing Strategy

sequenceDiagram
    participant Parser as Grammar Parser
    participant AST as TypeKind Enum
    participant Evaluator as Type Reader
    participant Operators as Operator Engine
    
    Parser->>AST: Parse "lequad" → TypeKind::Quad{Little, unsigned}
    AST->>Evaluator: read_quad(buffer, offset, Little, false)
    Evaluator->>Evaluator: Extract 8 bytes
    Evaluator->>Evaluator: LittleEndian::read_u64()
    Evaluator-->>AST: Value::Uint(u64)
    AST->>Operators: apply_operator(Equal, value, expected)
    Operators-->>Parser: Match result

Loading

Test Matrix

Type Variant	Endianness	Signedness	Test Data	Expected Value
quad	Native	Unsigned	[0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00]	Platform-dependent
lequad	Little	Unsigned	[0xEF,0xCD,0xAB,0x89,0x67,0x45,0x23,0x01]	0x0123456789ABCDEF
bequad	Big	Unsigned	[0x01,0x23,0x45,0x67,0x89,0xAB,0xCD,0xEF]	0x0123456789ABCDEF
lequad	Little	Signed	[0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF]	-1
bequad	Big	Signed	[0x7F,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF]	i64::MAX

Edge Cases to Test

1. Buffer boundaries: Reading quad at offset where only 7 bytes remain
2. Maximum values: u64::MAX, i64::MAX, i64::MIN
3. Zero values: All-zero 8-byte sequence
4. Endianness consistency: Same data read with different endianness produces different values
5. Operator compatibility: All operators (=, !=, &, &mask) work with quad values
6. Parser precedence: lequad matched before quad in parser

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase follows a consistent pattern for implementing integer types: `Byte` (8-bit), `Short` (16-bit), and `Long` (32-bit) are already implemented with endianness variants. The `TypeKind` enum in `file:src/parser/ast.rs` uses structured variants with `endian` and `signed` fields for multi-byte types. The `byteorder` crate provides `read_u64` methods for 64-bit reading. The `Value` enum already supports `u64` and `i64` through `Value::Uint` and `Value::Int`, and operators in `file:src/evaluator/operators.rs` already handle these types correctly.

## Approach

Following the established pattern for `Short` and `Long` types, implement the `Quad` variant as a structured enum with endianness and signedness support. The implementation will mirror the existing 16-bit and 32-bit implementations, using the `byteorder` crate's `read_u64` method for reading 8 bytes. The parser will recognize `quad`, `lequad`, and `bequad` type specifiers, mapping them to the appropriate `TypeKind::Quad` variant with corresponding endianness. This approach ensures consistency with the existing codebase and maintains compatibility with GNU file's magic format.

## Implementation Steps

### 1. Extend TypeKind Enum in AST

**File:** `file:src/parser/ast.rs`

Add the `Quad` variant to the `TypeKind` enum after the `Long` variant (around line 96):

```rust
/// 64-bit integer
Quad {
    /// Byte order
    endian: Endianness,
    /// Whether value is signed
    signed: bool,
},
```

Update the test section to include tests for the new `Quad` variant, following the pattern of existing `test_type_kind_short` and `test_type_kind_long` tests (around line 520-550).

### 2. Update Grammar Parser

**File:** `file:src/parser/grammar.rs`

In the `parse_type_and_operator` function (around line 1432), add quad type specifiers to the `alt()` combinator **before** the existing types to ensure longest-match-first parsing:

```rust
let (input, type_name) = alt((
    tag("lequad"),   // Add first
    tag("bequad"),   // Add second
    tag("quad"),     // Add third
    tag("lelong"),
    tag("belong"),
    // ... rest of existing types
))
```

In the match statement (around line 1458), add mappings for the quad types:

```rust
"quad" => TypeKind::Quad {
    endian: Endianness::Native,
    signed: false,
},
"lequad" => TypeKind::Quad {
    endian: Endianness::Little,
    signed: false,
},
"bequad" => TypeKind::Quad {
    endian: Endianness::Big,
    signed: false,
},
```

Add comprehensive tests in the test section (around line 1809-1879) following the pattern of `test_parse_type_endianness`:

- Test parsing `quad`, `lequad`, `bequad`
- Test with whitespace
- Test with remaining input
- Test in complete magic rules

### 3. Implement read_quad Function

**File:** `file:src/evaluator/types.rs`

Add the `read_quad` function after `read_long` (around line 207), following the exact pattern of `read_short` and `read_long`:

**Function signature:**
```rust
pub fn read_quad(
    buffer: &[u8],
    offset: usize,
    endian: Endianness,
    signed: bool,
) -> Result<Value, TypeReadError>
```

**Implementation details:**
- Extract 8 bytes using `buffer.get(offset..offset + 8)`
- Return `TypeReadError::BufferOverrun` if insufficient bytes
- Use `LittleEndian::read_u64`, `BigEndian::read_u64`, or `NativeEndian::read_u64` based on endianness
- For signed values, cast `u64` to `i64` and return `Value::Int(value as i64)` with `#[allow(clippy::cast_possible_wrap)]`
- For unsigned values, return `Value::Uint(value)`

**Documentation:** Include comprehensive doc comments with examples showing:
- Reading unsigned little-endian quad
- Reading signed values
- Buffer overrun error cases

### 4. Update read_typed_value Function

**File:** `file:src/evaluator/types.rs`

In the `read_typed_value` function (around line 307-358), add a new match arm for `TypeKind::Quad`:

```rust
TypeKind::Quad { endian, signed } => read_quad(buffer, offset, *endian, *signed),
```

This should be added after the `TypeKind::Long` arm and before `TypeKind::String`.

### 5. Verify Operator Support

**File:** `file:src/evaluator/operators.rs`

Review the operator implementations to confirm they already handle `u64` and `i64` values correctly:

- `apply_equal` (lines 9-66): Already handles `Value::Uint` and `Value::Int` with cross-type coercion
- `apply_not_equal` (lines 68-107): Delegates to `apply_equal`
- `apply_bitwise_and` (lines 109-163): Already handles bitwise operations on `u64` and `i64`
- `apply_operator` (lines 165-237): Handles `BitwiseAndMask` with `u64` values

**No changes required** - operators already support 64-bit values through the existing `Value::Uint(u64)` and `Value::Int(i64)` variants.

### 6. Add Unit Tests

**File:** `file:src/evaluator/types.rs`

Add comprehensive unit tests in the test module (after line 360), following the pattern of `read_short` and `read_long` tests:

**Test coverage:**
- `test_read_quad_little_endian_unsigned`: Test reading `0x0123456789ABCDEF` in little-endian
- `test_read_quad_big_endian_unsigned`: Test reading in big-endian
- `test_read_quad_native_endian_unsigned`: Test native endianness
- `test_read_quad_little_endian_signed`: Test signed interpretation with positive values
- `test_read_quad_signed_negative`: Test negative values (e.g., `0xFFFFFFFFFFFFFFFF` as -1)
- `test_read_quad_buffer_overrun`: Test error when buffer has fewer than 8 bytes
- `test_read_quad_at_buffer_boundary`: Test reading at exact buffer end
- `test_read_quad_max_values`: Test `u64::MAX` and `i64::MAX`/`i64::MIN`
- `test_read_quad_zero`: Test reading zero value

**File:** `file:src/parser/grammar.rs`

Add parser tests in the test section (around line 1880):

- `test_parse_quad_types`: Test parsing all three quad variants
- `test_parse_quad_with_operator`: Test `lequad&0xFFFFFFFF00000000`
- `test_parse_magic_rule_with_quad`: Test complete magic rules using quad types

### 7. Add Integration Tests

**File:** `file:tests/integration_tests.rs`

Add integration tests demonstrating real-world usage:

**Test scenarios:**
- Create a magic rule that detects 64-bit file signatures (e.g., ELF64 class field)
- Test reading 64-bit timestamps or file sizes
- Test with actual binary data containing 64-bit values
- Test endianness variants with known binary formats

**Example test structure:**
```rust
#[test]
fn test_quad_type_detection() {
    // Create magic file with quad rule
    // Test with buffer containing 64-bit value
    // Verify correct detection and value interpretation
}
```

### 8. Overflow Safety Verification

**Files:** `file:src/evaluator/types.rs`, `file:src/evaluator/operators.rs`

**Verify overflow-safe arithmetic:**
- The `read_quad` function uses direct casting from `u64` to `i64` for signed interpretation, which is safe as it preserves bit patterns
- Operators use bitwise operations on `u64` values, which don't overflow
- Comparison operators use type-safe comparisons without arithmetic
- The `Value` enum already stores `u64` and `i64` directly, avoiding intermediate conversions

**No additional overflow protection needed** - the implementation follows Rust's safe casting patterns and the existing codebase's approach.

### 9. Documentation Updates

**File:** `file:src/parser/grammar.rs`

Update the doc comment for `parse_type_and_operator` (around line 1402-1423) to include quad types:

```rust
/// - `quad` - 64-bit integer (native endian)
/// - `lequad` - 64-bit little-endian integer
/// - `bequad` - 64-bit big-endian integer
```

**File:** `file:src/evaluator/types.rs`

Update the module-level documentation (lines 1-5) to mention 64-bit integer support.

## Testing Strategy

```mermaid
sequenceDiagram
    participant Parser as Grammar Parser
    participant AST as TypeKind Enum
    participant Evaluator as Type Reader
    participant Operators as Operator Engine
    
    Parser->>AST: Parse "lequad" → TypeKind::Quad{Little, unsigned}
    AST->>Evaluator: read_quad(buffer, offset, Little, false)
    Evaluator->>Evaluator: Extract 8 bytes
    Evaluator->>Evaluator: LittleEndian::read_u64()
    Evaluator-->>AST: Value::Uint(u64)
    AST->>Operators: apply_operator(Equal, value, expected)
    Operators-->>Parser: Match result
```

### Test Matrix

| Type Variant | Endianness | Signedness | Test Data | Expected Value |
|--------------|------------|------------|-----------|----------------|
| `quad` | Native | Unsigned | `[0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00]` | Platform-dependent |
| `lequad` | Little | Unsigned | `[0xEF,0xCD,0xAB,0x89,0x67,0x45,0x23,0x01]` | `0x0123456789ABCDEF` |
| `bequad` | Big | Unsigned | `[0x01,0x23,0x45,0x67,0x89,0xAB,0xCD,0xEF]` | `0x0123456789ABCDEF` |
| `lequad` | Little | Signed | `[0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF]` | `-1` |
| `bequad` | Big | Signed | `[0x7F,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF]` | `i64::MAX` |

### Edge Cases to Test

1. **Buffer boundaries:** Reading quad at offset where only 7 bytes remain
2. **Maximum values:** `u64::MAX`, `i64::MAX`, `i64::MIN`
3. **Zero values:** All-zero 8-byte sequence
4. **Endianness consistency:** Same data read with different endianness produces different values
5. **Operator compatibility:** All operators (=, !=, &, &mask) work with quad values
6. **Parser precedence:** `lequad` matched before `quad` in parser

---

Execution Information

Branch: main
Commit: 36adac5

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.