Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
a238974
feat(operators): implement comparison operators and their evaluations
unclesp1d3r Mar 1, 2026
9fbfd88
feat(operators): add comparison operators for evaluation
unclesp1d3r Mar 1, 2026
7d34c66
feat(parser): add signed byte type and unsigned variants
unclesp1d3r Mar 1, 2026
252b292
feat(types): support signed/unsigned byte reads
unclesp1d3r Mar 1, 2026
2ef57a4
feat(evaluator): adopt signedness and extend comparisons
unclesp1d3r Mar 1, 2026
6a3ed09
chore(build): update serialization, docs, and builtins for unsigned t…
unclesp1d3r Mar 1, 2026
525bd32
test(evaluator): add tests for comparison operators in rules
unclesp1d3r Mar 1, 2026
9c9ab52
test(types): enhance byte reading tests for edge cases and signedness
unclesp1d3r Mar 1, 2026
9258448
docs: updates for PR #104 (#105)
dosubot[bot] Mar 1, 2026
aae86fe
docs(agents): update architecture constraints and testing strategy
unclesp1d3r Mar 1, 2026
7e90934
test(evaluator): add tests for comparison operators and strengthen se…
unclesp1d3r Mar 1, 2026
6370879
chore(deps): add commit message prefixes for dependabot updates
unclesp1d3r Mar 1, 2026
ec3a088
test(evaluator): add tests for signed and unsigned byte comparison
unclesp1d3r Mar 1, 2026
6520891
docs(evaluator): update evaluation context and operator application d…
unclesp1d3r Mar 1, 2026
631b231
fix(evaluator): improve error handling for rule evaluation
unclesp1d3r Mar 1, 2026
e78e109
docs(evaluator): clarify comparison operators for numeric and lexicog…
unclesp1d3r Mar 1, 2026
cf22db1
fix(mergify): correct regex for conventional commit title validation
unclesp1d3r Mar 1, 2026
aa3a5cd
feat(evaluator): implement value coercion for comparison operators
unclesp1d3r Mar 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,21 @@ updates:
schedule:
interval: "weekly"
rebase-strategy: "disabled"
commit-message:
prefix: "chore(deps)"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
rebase-strategy: "disabled"
commit-message:
prefix: "chore(deps)"

- package-ecosystem: "devcontainers"
directory: "/"
schedule:
interval: "weekly"
rebase-strategy: "disabled"
commit-message:
prefix: "chore(deps)"
53 changes: 41 additions & 12 deletions .mergify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,47 +11,69 @@ queue_rules:
- check-success = coverage

pull_request_rules:
# Tier 1: Maintainer PRs -- queue when maintainer adds 'lgtm' label
- name: Queue maintainer PRs with lgtm label
# Tier 1: Trusted bot PRs -- auto-approve and queue immediately
- name: Auto-approve and queue dependabot PRs
conditions:
- base = main
- "author=@maintainers"
- label = lgtm
- author = dependabot[bot]
- -draft
- label != do-not-merge
# release.yml is autogenerated by cargo-dist -- dependabot updates to
# pinned actions in it will break the release pipeline. Dependabot has no
# way to ignore specific workflow files, so we block it here instead.
- -files~=\.github/workflows/release\.yml
actions:
review:
type: APPROVE
message: Automatically approved by Mergify
queue:
name: default

Comment on lines +14 to +31
Copy link

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR description focuses on parser type/operator support, but it also changes merge automation (Mergify rules/merge protections) and Dependabot configuration. These CI/ops changes are significant and should either be called out explicitly in the PR description or split into a separate PR to keep review/rollback risk isolated.

Copilot uses AI. Check for mistakes.
- name: Auto-approve and queue dosu PRs
conditions:
- base = main
- author = dosubot[bot]
- -draft
- label != do-not-merge
actions:
review:
type: APPROVE
message: Automatically approved by Mergify
queue:
name: default

# Tier 2: Trusted bot PRs -- auto-queue when checks pass
- name: Auto-queue release-plz PRs
conditions:
- base = main
- "head ~= ^release-plz-"
- -draft
- label != do-not-merge
actions:
queue:
name: default

- name: Auto-approve and queue dependabot PRs
# Tier 2: Maintainer PRs -- queue when maintainer self-labels 'lgtm'
# (no approval required; solves the sole-maintainer self-merge problem)
- name: Queue maintainer PRs with lgtm label
conditions:
- base = main
- author = dependabot[bot]
- "author=@maintainers"
- -draft
- label = lgtm
- label != do-not-merge
- -files~=\.github/workflows/release\.yml
actions:
review:
type: APPROVE
message: Automatically approved by Mergify
queue:
name: default

# Tier 3: All other PRs (external contributors, copilot) -- require maintainer approval
# Tier 3: External contributor PRs -- require maintainer approval
- name: Queue external PRs when approved by maintainer
conditions:
- base = main
- "-author=@maintainers"
- author != dependabot[bot]
- author != dosubot[bot]
- "-head ~= ^release-plz-"
- -draft
- "approved-reviews-by=@maintainers"
- label != do-not-merge
actions:
Expand All @@ -69,6 +91,13 @@ pull_request_rules:
update: {}

merge_protections:
- name: Enforce conventional commit
description: Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
if:
- base = main
Copy link

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new "Enforce conventional commit" merge protection applies to all PRs to main, but it validates the PR title. This will likely block auto-queued PRs from dependabot/release-plz (their titles typically don’t follow feat(scope): ...), meaning the bot rules above can approve+queue but merges will still be prevented. Consider excluding trusted bots/release-plz branches from this protection (or enforcing conventional commits via a CI check that validates commits instead of PR titles).

Suggested change
- base = main
- base = main
- author != dependabot[bot]
- "-head ~= ^release-plz/"

Copilot uses AI. Check for mistakes.
success_conditions:
- "title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\\\\(.+\\\\))?!?:"

- name: CI must pass
description: >-
All CI checks must pass. This protection prevents manual merges
Expand Down
21 changes: 13 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Target File → Memory Mapper → File Buffer
// Core data structures in lib.rs
pub struct MagicRule { /* ... */ }
pub enum TypeKind {
Byte,
Byte { signed: bool },
Short { endian: Endianness, signed: bool },
Long { endian: Endianness, signed: bool },
String { max_length: Option<usize> },
Expand Down Expand Up @@ -138,10 +138,13 @@ pub fn evaluate_magic_rules(

- `src/error.rs` is shared with `build.rs` -- cannot reference lib-only types like `crate::io::IoError`
- `FileError(String)` wraps structured I/O errors as strings to work around the build.rs constraint
- `build.rs` and `src/build_helpers.rs` have duplicate `serialize_*` functions -- both must be updated when adding enum variants
- Use `ParseError::IoError` for I/O errors in parser code, not `ParseError::invalid_syntax`
- Use `LibmagicError::ConfigError` for config validation, not `ParseError::invalid_syntax`
- Clippy pedantic lints are active (e.g., prefer `trailing_zeros()` over bitwise masks)
- All public enum variants need `# Examples` rustdoc sections
- Comparison operators share a `compare_values() -> Option<Ordering>` helper in `operators.rs` -- new comparison logic goes there, not in individual `apply_*` functions
- libmagic types are signed by default (`byte`, `short`, `long`); unsigned variants use `u` prefix (`ubyte`, `ushort`, `ulong`, etc.)

### Naming Conventions

Expand Down Expand Up @@ -180,20 +183,20 @@ cargo test --doc # Test documentation examples
- **Property Tests**: Use `proptest` for fuzzing magic rule evaluation
- **Benchmarks**: Critical path performance tests with `criterion`
- **Coverage**: Target >85% with `cargo llvm-cov`
- **Test style**: Prefer table-driven tests over one-assertion-per-function tests; consolidate related cases into a single test with descriptive failure messages

## Magic File Compatibility

### Currently Implemented (v0.1.0)

- **Offsets**: Absolute and from-end specifications (indirect and relative are parsed but not yet evaluated)
- **Types**: `byte`, `short`, `long`, `string` with endianness support
- **Operators**: `=` (equal), `!=` (not equal), `&` (bitwise AND with optional mask)
- **Types**: `byte`, `short`, `long`, `string` with endianness support; unsigned variants `ubyte`, `ushort`/`ubeshort`/`uleshort`, `ulong`/`ubelong`/`ulelong`; types are signed by default (libmagic-compatible)
- **Operators**: `=` (equal), `!=` (not equal), `<` (less than), `>` (greater than), `<=` (less equal), `>=` (greater equal), `&` (bitwise AND with optional mask)
- **Nested Rules**: Hierarchical rule evaluation with proper indentation
- **String Matching**: Exact string matching with null-termination

### Planned Features (v1.0+)

- Comparison operators: `>`, `<`, `>=`, `<=`
- Bitwise XOR operator: `^`
- Regex type: Pattern matching with binary-safe regex support
- Additional types: 64-bit integers, floats, doubles, dates
Expand Down Expand Up @@ -226,7 +229,6 @@ impl BinaryRegex for regex::bytes::Regex {

### Operators

- No comparison operators (`>`, `<`, `>=`, `<=`)
- No XOR operator (`^`)
- No negation operator (`~`)
- BitwiseAnd supports mask values but not all libmagic mask syntax
Expand Down Expand Up @@ -316,13 +318,16 @@ sample.bin: ELF 64-bit LSB executable, x86-64, version 1 (SYSV)

### Adding New Operators

> **Note:** Currently implemented operators are `Equal`, `NotEqual`, and `BitwiseAnd` (with `BitwiseAndMask`). Comparison operators (`>`, `<`) and XOR (`^`) are planned for future releases.
> **Note:** Currently implemented operators are `Equal`, `NotEqual`, `LessThan`, `GreaterThan`, `LessEqual`, `GreaterEqual`, and `BitwiseAnd` (with `BitwiseAndMask`). XOR (`^`) is planned for future releases.

1. Extend `Operator` enum in `src/parser/ast.rs`
2. Add parsing logic in `src/parser/grammar.rs`
3. Implement operator logic in `src/evaluator/operators.rs`
4. Add tests for the new operator
5. Update documentation
4. Update `serialize_operator()` in both `src/build_helpers.rs` AND `build.rs` (they have duplicate match statements)
5. Update strength calculation match in `src/evaluator/strength.rs`
6. Update `arb_operator()` in `tests/property_tests.rs`
7. Add tests for the new operator
8. Update documentation

### Performance Optimization

Expand Down
15 changes: 13 additions & 2 deletions build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ fn serialize_offset_spec(offset: &OffsetSpec) -> String {

fn serialize_type_kind(typ: &TypeKind) -> String {
match typ {
TypeKind::Byte => "TypeKind::Byte".to_string(),
TypeKind::Byte { signed } => format!("TypeKind::Byte {{ signed: {signed} }}"),
TypeKind::Short { endian, signed } => format!(
"TypeKind::Short {{ endian: {}, signed: {} }}",
serialize_endianness(*endian),
Expand All @@ -293,6 +293,10 @@ fn serialize_operator(op: &Operator) -> String {
match op {
Operator::Equal => "Operator::Equal".to_string(),
Operator::NotEqual => "Operator::NotEqual".to_string(),
Operator::LessThan => "Operator::LessThan".to_string(),
Operator::GreaterThan => "Operator::GreaterThan".to_string(),
Operator::LessEqual => "Operator::LessEqual".to_string(),
Operator::GreaterEqual => "Operator::GreaterEqual".to_string(),
Operator::BitwiseAnd => "Operator::BitwiseAnd".to_string(),
Operator::BitwiseAndMask(mask) => format!("Operator::BitwiseAndMask({mask})"),
}
Expand All @@ -301,7 +305,14 @@ fn serialize_operator(op: &Operator) -> String {
fn serialize_value(value: &Value) -> String {
match value {
Value::Uint(number) => format!("Value::Uint({})", format_number(*number)),
Value::Int(number) => format!("Value::Int({})", format_number(*number as u64)),
Value::Int(number) => {
if *number < 0 {
let abs = number.unsigned_abs();
format!("Value::Int(-{})", format_number(abs))
} else {
format!("Value::Int({})", format_number(*number as u64))
}
}
Value::Bytes(bytes) => format!("Value::Bytes({})", format_byte_vec(bytes)),
Value::String(text) => format!(
"Value::String(String::from({}))",
Expand Down
14 changes: 9 additions & 5 deletions docs/API_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ use libmagic_rs::TypeKind;

| Variant | Description |
|---------|-------------|
| `Byte` | Single byte |
| `Byte { signed }` | Single byte with explicit signedness |
| `Short { endian, signed }` | 16-bit integer |
| `Long { endian, signed }` | 32-bit integer |
| `String { max_length }` | String data |
Expand All @@ -313,10 +313,14 @@ use libmagic_rs::Operator;

| Variant | Description |
|---------|-------------|
| `Equal` | Equality comparison |
| `NotEqual` | Inequality comparison |
| `BitwiseAnd` | Bitwise AND |
| `BitwiseAndMask(u64)` | Bitwise AND with mask |
| `Equal` | Equality comparison (`=` or `==`) |
| `NotEqual` | Inequality comparison (`!=` or `<>`) |
| `LessThan` | Less than comparison (`<`) |
| `GreaterThan` | Greater than comparison (`>`) |
| `LessEqual` | Less than or equal comparison (`<=`) |
| `GreaterEqual` | Greater than or equal comparison (`>=`) |
| `BitwiseAnd` | Bitwise AND (`&`) |
| `BitwiseAndMask(u64)` | Bitwise AND with mask value |

#### Value

Expand Down
20 changes: 17 additions & 3 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ libmagic-rs/
│ │
│ ├── parser/ # Magic file parsing
│ │ ├── mod.rs # Parser interface, file loading
│ │ ├── ast.rs # AST definitions (MagicRule, TypeKind, etc.)
│ │ ├── ast.rs # AST definitions (MagicRule, TypeKind::Byte { signed: bool }, etc.)
│ │ └── grammar.rs # nom-based parsing combinators
│ │
│ ├── evaluator/ # Rule evaluation engine
Expand Down Expand Up @@ -272,6 +272,9 @@ pub struct MagicRule {
- Child rules are evaluated only if parent matches
- Deeper matches = higher confidence

**Operator Support:**
- Supports comparison operators (`<`, `>`, `<=`, `>=`) in addition to equality (`=`, `!=`) and bitwise operators (`&`)

### EvaluationContext

Tracks state during rule evaluation.
Expand Down Expand Up @@ -472,8 +475,19 @@ The evaluation hot path is optimized for:
1. Add variant to `Operator` enum (`ast.rs`)
2. Add parsing logic (`grammar.rs`)
3. Add comparison logic (`operators.rs`)
4. Add tests
5. Update documentation
4. Add serialization for build-time (`build.rs` and `build_helpers.rs`)
5. Add tests
6. Update documentation

**Implemented Operators:**
- `Equal` (`=`, `==`)
- `NotEqual` (`!=`, `<>`)
- `LessThan` (`<`)
- `GreaterThan` (`>`)
- `LessEqual` (`<=`)
- `GreaterEqual` (`>=`)
- `BitwiseAnd` (`&`)
- `BitwiseAndMask` (`&` with mask)

### Adding Output Formats

Expand Down
7 changes: 5 additions & 2 deletions docs/MAGIC_FORMAT.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,10 @@ Example:
|----------|-------------|---------|
| `=` | Equal (default) | `0 long =0xcafebabe` |
| `!` | Not equal | `4 byte !0` |
| `>` | Greater than | `8 long >1000` |
| `<` | Less than | `8 long <100` |
| `>` | Greater than | `8 long >1000` |
| `<=` | Less than or equal | `8 long <=100` |
| `>=` | Greater than or equal | `8 long >=1000` |
| `&` | Bitwise AND | `4 byte &0x80` |
| `^` | Bitwise XOR | `4 byte ^0xff` |

Expand Down Expand Up @@ -469,7 +471,7 @@ Consider:
- Indirect offsets (basic)
- Byte, short, long types
- String type
- Equal, not-equal operators
- Comparison operators (`=`, `!`, `<`, `>`, `<=`, `>=`)
- Bitwise AND operator
- Nested rules
- Comments
Expand All @@ -484,6 +486,7 @@ Consider:

### Recently Added

- **Comparison operators**: Full support for `<`, `>`, `<=`, `>=` operators
- **Strength modifiers**: The `!:strength` directive for adjusting rule priority

---
Expand Down
10 changes: 7 additions & 3 deletions docs/src/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ use libmagic_rs::TypeKind;

| Variant | Description |
|---------|-------------|
| `Byte` | Single byte |
| `Byte { signed }` | Single byte with explicit signedness |
| `Short { endian, signed }` | 16-bit integer |
| `Long { endian, signed }` | 32-bit integer |
| `String { max_length }` | String data |
Expand All @@ -238,8 +238,12 @@ use libmagic_rs::Operator;

| Variant | Description |
|---------|-------------|
| `Equal` | Equality comparison |
| `NotEqual` | Inequality comparison |
| `Equal` | Equality comparison (`=`) |
| `NotEqual` | Inequality comparison (`!=`) |
| `LessThan` | Less than comparison (`<`) |
| `GreaterThan` | Greater than comparison (`>`) |
| `LessEqual` | Less than or equal comparison (`<=`) |
| `GreaterEqual` | Greater than or equal comparison (`>=`) |
| `BitwiseAnd` | Bitwise AND |
| `BitwiseAndMask(u64)` | Bitwise AND with mask |

Expand Down
Loading
Loading