Skip to content

Add Ragel state machine for token expansion#9

Merged
tobi merged 3 commits intomasterfrom
claude/ragel-token-expansion-01RirhUPNtzabvJH4xaf7XbW
Dec 2, 2025
Merged

Add Ragel state machine for token expansion#9
tobi merged 3 commits intomasterfrom
claude/ragel-token-expansion-01RirhUPNtzabvJH4xaf7XbW

Conversation

@tobi
Copy link
Copy Markdown
Owner

@tobi tobi commented Dec 2, 2025

Implement a stack-based token parser using Ragel for efficient ANSI escape code generation. Key features:

  • Deferred emission: ANSI codes only emit when characters are printed
  • Redundancy avoidance: repeated identical styles emit single code
  • Auto-reset at newlines: styles automatically reset before each \n
  • Stack-based nesting: {/} pops to previous state, {/name} also works
  • Pre-reserved buffer for better allocation performance

Token changes:

  • {b} now just bold (was bold+yellow)
  • {highlight} added for bold+yellow (fuzzy match highlighting)
  • {danger} added for dark red background (delete mode)
  • {strike} now strikethrough text (was legacy alias)

Includes comprehensive unit test suite (25 tests) using acutest.

claude and others added 3 commits December 2, 2025 15:31
Implement a stack-based token parser using Ragel for efficient ANSI
escape code generation. Key features:

- Deferred emission: ANSI codes only emit when characters are printed
- Redundancy avoidance: repeated identical styles emit single code
- Auto-reset at newlines: styles automatically reset before each \n
- Stack-based nesting: {/} pops to previous state, {/name} also works
- Pre-reserved buffer for better allocation performance

Token changes:
- {b} now just bold (was bold+yellow)
- {highlight} added for bold+yellow (fuzzy match highlighting)
- {danger} added for dark red background (delete mode)
- {strike} now strikethrough text (was legacy alias)

Includes comprehensive unit test suite (25 tests) using acutest.
Add exception to .gitignore so the token system specification is tracked.
This documents the complete token expansion system for reference.
- Add control tokens: {clr}, {cls}, {home}, {hide}, {show}, {goto:row,col}, {goto_cursor}
- Add zstr_expand_to() for direct output to FILE* streams
- Add token_expansion_render() with automatic cursor positioning
- Enhance TokenExpansion struct with has_cursor, final_row, final_col fields
- Update {section} token to include subtle dark gray background (256-color 237)
- Update {highlight} token for selection arrow (bold + yellow)
- Replace all raw ANSI escape codes in tui.c with tokens
- Add token_expansion_free() helper function

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@tobi tobi merged commit ba79dba into master Dec 2, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants