Parser: implement date and timestamp types

[unclesp1d3r]

Summary

Implement date/timestamp type specifiers: date, ldate, bedate, beldate, ledate, leldate, qdate, qldate, beqdate, beqldate, leqdate, leqldate, medate, meldate.

Context

Date types interpret byte sequences as Unix timestamps (seconds since epoch) and format them for display. 32-bit and 64-bit variants exist with various endianness options. date types use UTC; ldate types use local time.

Current state: No date types exist in TypeKind or the parser.

Acceptance Criteria

• TypeKind extended with Date and QDate variants (32-bit and 64-bit)
• Parser recognizes all date type specifiers with endianness
• Evaluator reads timestamps and formats as human-readable dates
• UTC vs local time distinction honored
• Comparison operators work with date values (comparing timestamps)
• Unit tests for each variant

Impact

MEDIUM - Used in archive formats (tar, cpio), filesystem images, and document metadata for creation/modification timestamps.

Files to Modify

• src/parser/ast.rs - Extend TypeKind enum
• src/parser/grammar.rs - Parse date specifiers
• src/evaluator/types.rs - Implement timestamp reading and formatting

References

• GNU file magic(5) man page
• Epic Epic: libmagic-rs Phase 1 MVP #24

[traycerai]

Plan

Observations

The codebase currently supports basic numeric types (Byte, Short, Long) and String types with endianness variants. The TypeKind enum in file:src/parser/ast.rs uses a pattern where endianness is a field within type variants. The parser in file:src/parser/grammar.rs uses nom combinators with an alt combinator to match type names, and the evaluator in file:src/evaluator/types.rs dispatches type reading through read_typed_value. No date/time handling exists, and no chrono or similar library is currently used. The codebase has comprehensive test coverage and follows strict linting rules.

Approach

Implement date/timestamp types by extending the existing type system architecture. Add Date (32-bit) and QDate (64-bit) variants to TypeKind with fields for endianness and local time flag. Extend the parser to recognize all 14 date type specifiers by adding them to the type name matching logic. Implement timestamp reading functions following the pattern of existing read_short and read_long functions. Add the chrono crate for timestamp-to-date conversion and formatting. Store timestamps as Value::Int for signed interpretation, enabling comparison operators to work naturally on the underlying timestamp values.

Implementation Steps

1. Add chrono dependency

Add chrono to file:Cargo.toml dependencies section:

chrono = "0.4"

2. Extend TypeKind enum in file:src/parser/ast.rs

Add two new variants to the TypeKind enum after the existing Long variant:

• Date variant with fields:

  • endian: Endianness - byte order (Native, Little, Big, or Middle-endian)
  • local_time: bool - true for ldate types (local time), false for date types (UTC)

• QDate variant with fields:

  • endian: Endianness - byte order
  • local_time: bool - true for qldate types, false for qdate types

Add a Middle variant to the Endianness enum for middle-endian (PDP-11) byte order used by medate and meldate types.

3. Extend parser in file:src/parser/grammar.rs

In the parse_type_and_operator function, extend the alt combinator to include all date type specifiers. Add these tags in order of longest-first to prevent prefix matching issues:

64-bit date types (qdate variants):

• "beqldate" → QDate { endian: Big, local_time: true }
• "leqldate" → QDate { endian: Little, local_time: true }
• "beqdate" → QDate { endian: Big, local_time: false }
• "leqdate" → QDate { endian: Little, local_time: false }
• "qldate" → QDate { endian: Native, local_time: true }
• "qdate" → QDate { endian: Native, local_time: false }

32-bit date types (date variants):

• "beldate" → Date { endian: Big, local_time: true }
• "meldate" → Date { endian: Middle, local_time: true }
• "leldate" → Date { endian: Little, local_time: true }
• "bedate" → Date { endian: Big, local_time: false }
• "medate" → Date { endian: Middle, local_time: false }
• "ledate" → Date { endian: Little, local_time: false }
• "ldate" → Date { endian: Native, local_time: true }
• "date" → Date { endian: Native, local_time: false }

Place these before existing type tags in the alt combinator to ensure proper matching.

4. Implement timestamp reading in file:src/evaluator/types.rs

Add helper function for middle-endian byte order conversion:

• read_middle_endian_u32 - reads 4 bytes in PDP-11 byte order (swap 16-bit words)

Add timestamp reading functions following the pattern of read_short and read_long:

For 32-bit dates:

• read_date(buffer, offset, endian, local_time) function that:
  • Reads 4 bytes from buffer at offset
  • Converts bytes to u32 using specified endianness (including Middle)
  • Casts to i64 for signed timestamp interpretation
  • Uses chrono to convert Unix timestamp to DateTime
  • Formats as human-readable string (e.g., "Wed Feb 14 12:34:56 2026")
  • Returns Value::String with formatted date
  • Handles local_time flag by using Local::timestamp vs Utc::timestamp

For 64-bit dates:

• read_qdate(buffer, offset, endian, local_time) function that:
  • Reads 8 bytes from buffer at offset
  • Converts bytes to i64 using specified endianness
  • Uses chrono to convert Unix timestamp to DateTime
  • Formats as human-readable string
  • Returns Value::String with formatted date
  • Handles local_time flag appropriately

Update read_typed_value function to add match arms for:

• TypeKind::Date { endian, local_time } → call read_date
• TypeKind::QDate { endian, local_time } → call read_qdate

5. Handle comparison operators

Date values will be stored as formatted strings in Value::String. For numeric comparisons on timestamps:

Consider adding a Value::Timestamp(i64) variant to preserve the numeric timestamp value for comparisons, or store both the timestamp and formatted string. This allows:

• Equality/inequality operators to work on formatted strings
• Numeric comparison operators (if added later) to work on timestamp values

Alternatively, for the initial implementation, comparisons can work on the formatted string representation, which is sufficient for equality checks.

6. Add comprehensive unit tests

In file:src/parser/ast.rs tests:

• Test Date and QDate variant creation with all endianness options
• Test serialization/deserialization of date types
• Test equality comparisons between date type variants

In file:src/parser/grammar.rs tests:

• Test parsing each of the 14 date type specifiers
• Test that date types are correctly distinguished from similar prefixes
• Test date types with whitespace and operators
• Test complete magic rules using date types

In file:src/evaluator/types.rs tests:

• Test read_date with various endianness options (Native, Little, Big, Middle)
• Test read_qdate with various endianness options
• Test UTC vs local time formatting
• Test buffer overrun errors for date types
• Test known timestamp values (e.g., Unix epoch, Y2K, specific dates)
• Test edge cases (negative timestamps, far future dates)
• Test middle-endian byte order conversion
• Test read_typed_value dispatching for Date and QDate types

Integration tests:

• Create test magic rules with date types
• Test evaluation of files with known timestamps
• Test hierarchical rules with date types
• Test comparison operators with date values

7. Update documentation

Add documentation comments for:

• New Date and QDate variants in TypeKind enum
• New Middle variant in Endianness enum
• read_date and read_qdate functions
• Middle-endian conversion helper function

Include examples showing:

• Parsing date type specifiers
• Reading timestamps from byte buffers
• UTC vs local time distinction
• Format of human-readable date output

Date Formatting Specification

Use chrono's formatting to produce output compatible with GNU file:

• Format: "%a %b %e %H:%M:%S %Y" (e.g., "Wed Feb 14 12:34:56 2026")
• For UTC dates: use chrono::Utc::timestamp_opt
• For local dates: use chrono::Local::timestamp_opt
• Handle invalid timestamps gracefully (return error or special string)

Middle-Endian Byte Order

PDP-11 middle-endian format for 32-bit values:

• Bytes stored as: [byte1, byte0, byte3, byte2]
• Read as two 16-bit little-endian words, then swap the words
• Example: 0x12345678 stored as [0x34, 0x12, 0x78, 0x56]

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase currently supports basic numeric types (Byte, Short, Long) and String types with endianness variants. The `TypeKind` enum in `file:src/parser/ast.rs` uses a pattern where endianness is a field within type variants. The parser in `file:src/parser/grammar.rs` uses nom combinators with an `alt` combinator to match type names, and the evaluator in `file:src/evaluator/types.rs` dispatches type reading through `read_typed_value`. No date/time handling exists, and no chrono or similar library is currently used. The codebase has comprehensive test coverage and follows strict linting rules.

## Approach

Implement date/timestamp types by extending the existing type system architecture. Add `Date` (32-bit) and `QDate` (64-bit) variants to `TypeKind` with fields for endianness and local time flag. Extend the parser to recognize all 14 date type specifiers by adding them to the type name matching logic. Implement timestamp reading functions following the pattern of existing `read_short` and `read_long` functions. Add the `chrono` crate for timestamp-to-date conversion and formatting. Store timestamps as `Value::Int` for signed interpretation, enabling comparison operators to work naturally on the underlying timestamp values.

## Implementation Steps

### 1. Add chrono dependency

Add `chrono` to `file:Cargo.toml` dependencies section:
```toml
chrono = "0.4"
```

### 2. Extend TypeKind enum in file:src/parser/ast.rs

Add two new variants to the `TypeKind` enum after the existing `Long` variant:

- `Date` variant with fields:
  - `endian: Endianness` - byte order (Native, Little, Big, or Middle-endian)
  - `local_time: bool` - true for ldate types (local time), false for date types (UTC)
  
- `QDate` variant with fields:
  - `endian: Endianness` - byte order
  - `local_time: bool` - true for qldate types, false for qdate types

Add a `Middle` variant to the `Endianness` enum for middle-endian (PDP-11) byte order used by `medate` and `meldate` types.

### 3. Extend parser in file:src/parser/grammar.rs

In the `parse_type_and_operator` function, extend the `alt` combinator to include all date type specifiers. Add these tags in order of longest-first to prevent prefix matching issues:

**64-bit date types (qdate variants):**
- `"beqldate"` → `QDate { endian: Big, local_time: true }`
- `"leqldate"` → `QDate { endian: Little, local_time: true }`
- `"beqdate"` → `QDate { endian: Big, local_time: false }`
- `"leqdate"` → `QDate { endian: Little, local_time: false }`
- `"qldate"` → `QDate { endian: Native, local_time: true }`
- `"qdate"` → `QDate { endian: Native, local_time: false }`

**32-bit date types (date variants):**
- `"beldate"` → `Date { endian: Big, local_time: true }`
- `"meldate"` → `Date { endian: Middle, local_time: true }`
- `"leldate"` → `Date { endian: Little, local_time: true }`
- `"bedate"` → `Date { endian: Big, local_time: false }`
- `"medate"` → `Date { endian: Middle, local_time: false }`
- `"ledate"` → `Date { endian: Little, local_time: false }`
- `"ldate"` → `Date { endian: Native, local_time: true }`
- `"date"` → `Date { endian: Native, local_time: false }`

Place these before existing type tags in the `alt` combinator to ensure proper matching.

### 4. Implement timestamp reading in file:src/evaluator/types.rs

Add helper function for middle-endian byte order conversion:
- `read_middle_endian_u32` - reads 4 bytes in PDP-11 byte order (swap 16-bit words)

Add timestamp reading functions following the pattern of `read_short` and `read_long`:

**For 32-bit dates:**
- `read_date(buffer, offset, endian, local_time)` function that:
  - Reads 4 bytes from buffer at offset
  - Converts bytes to u32 using specified endianness (including Middle)
  - Casts to i64 for signed timestamp interpretation
  - Uses chrono to convert Unix timestamp to DateTime
  - Formats as human-readable string (e.g., "Wed Feb 14 12:34:56 2026")
  - Returns `Value::String` with formatted date
  - Handles local_time flag by using `Local::timestamp` vs `Utc::timestamp`

**For 64-bit dates:**
- `read_qdate(buffer, offset, endian, local_time)` function that:
  - Reads 8 bytes from buffer at offset
  - Converts bytes to i64 using specified endianness
  - Uses chrono to convert Unix timestamp to DateTime
  - Formats as human-readable string
  - Returns `Value::String` with formatted date
  - Handles local_time flag appropriately

Update `read_typed_value` function to add match arms for:
- `TypeKind::Date { endian, local_time }` → call `read_date`
- `TypeKind::QDate { endian, local_time }` → call `read_qdate`

### 5. Handle comparison operators

Date values will be stored as formatted strings in `Value::String`. For numeric comparisons on timestamps:

Consider adding a `Value::Timestamp(i64)` variant to preserve the numeric timestamp value for comparisons, or store both the timestamp and formatted string. This allows:
- Equality/inequality operators to work on formatted strings
- Numeric comparison operators (if added later) to work on timestamp values

Alternatively, for the initial implementation, comparisons can work on the formatted string representation, which is sufficient for equality checks.

### 6. Add comprehensive unit tests

**In file:src/parser/ast.rs tests:**
- Test Date and QDate variant creation with all endianness options
- Test serialization/deserialization of date types
- Test equality comparisons between date type variants

**In file:src/parser/grammar.rs tests:**
- Test parsing each of the 14 date type specifiers
- Test that date types are correctly distinguished from similar prefixes
- Test date types with whitespace and operators
- Test complete magic rules using date types

**In file:src/evaluator/types.rs tests:**
- Test `read_date` with various endianness options (Native, Little, Big, Middle)
- Test `read_qdate` with various endianness options
- Test UTC vs local time formatting
- Test buffer overrun errors for date types
- Test known timestamp values (e.g., Unix epoch, Y2K, specific dates)
- Test edge cases (negative timestamps, far future dates)
- Test middle-endian byte order conversion
- Test `read_typed_value` dispatching for Date and QDate types

**Integration tests:**
- Create test magic rules with date types
- Test evaluation of files with known timestamps
- Test hierarchical rules with date types
- Test comparison operators with date values

### 7. Update documentation

Add documentation comments for:
- New `Date` and `QDate` variants in `TypeKind` enum
- New `Middle` variant in `Endianness` enum
- `read_date` and `read_qdate` functions
- Middle-endian conversion helper function

Include examples showing:
- Parsing date type specifiers
- Reading timestamps from byte buffers
- UTC vs local time distinction
- Format of human-readable date output

## Date Formatting Specification

Use chrono's formatting to produce output compatible with GNU file:
- Format: `"%a %b %e %H:%M:%S %Y"` (e.g., "Wed Feb 14 12:34:56 2026")
- For UTC dates: use `chrono::Utc::timestamp_opt`
- For local dates: use `chrono::Local::timestamp_opt`
- Handle invalid timestamps gracefully (return error or special string)

## Middle-Endian Byte Order

PDP-11 middle-endian format for 32-bit values:
- Bytes stored as: [byte1, byte0, byte3, byte2]
- Read as two 16-bit little-endian words, then swap the words
- Example: `0x12345678` stored as `[0x34, 0x12, 0x78, 0x56]`

---

Execution Information

Branch: main
Commit: 36adac5

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.