Refactor segment_inline to use &str parsing (no Vec<char>)#255

Merged

leynos merged 7 commits intomainfrom

terragon/optimize-tokenize-markdown-x4e2p6

Nov 12, 2025

Commits on Nov 9, 2025

refactor(wrap/tokenize): use &str instead of &[char] to improve efficiency

leynos
and
terragon-labs[bot]
committed

Commits on Nov 11, 2025

refactor(wrap/tokenize): simplify advance_char function using map_or

leynos
and
terragon-labs[bot]
committed
docs(tokenizer): add architecture documentation describing inline tokenizer flow

leynos
and
terragon-labs[bot]
committed
refactor(wrap/tokenize): improve multibyte character handling and parsing logic

leynos
and
terragon-labs[bot]
committed
refactor(tokenize): extract backtick fence handling into a dedicated function

leynos
and
terragon-labs[bot]
committed
docs(architecture): fix typo by adding missing comma in architecture.md

leynos
and
terragon-labs[bot]
committed

Commits on Nov 12, 2025

docs(architecture): correct spelling of 'summarises' to 'summarizes' in documentation

leynos
and
terragon-labs[bot]
committed