Skip to content

fix: multiply markWord calls#235

Open
Shipy4kaRU wants to merge 1 commit intootakustay:masterfrom
Shipy4kaRU:fix/multiply-markworkd
Open

fix: multiply markWord calls#235
Shipy4kaRU wants to merge 1 commit intootakustay:masterfrom
Shipy4kaRU:fix/multiply-markworkd

Conversation

@Shipy4kaRU
Copy link

@Shipy4kaRU Shipy4kaRU commented Feb 23, 2026

Hi! First of all - thank you for this library; it’s really one of the most capable and pleasant to use: rich feature set and a clean API.

This PR fixes a bug when markWord is used multiple times (e.g. several enhancers in a row). In your tokenization example it was failing because after the first markWord run, the leaf node in some paths became a node of type mark instead of text. On the next markWord run, leafOf(path) was called for those paths; leafOf expects a text node and throws otherwise (Invalid token path with leaf of type mark). That’s why tokenization didn’t complete in your own example. The fix: if the path’s leaf node is already non-text (e.g. a mark), we return the path unchanged and don’t call leafOf, so multiple markWord enhancers can be chained without errors

P.S: maybe it can fix #234 issue

UPD: add logs
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Syntax highlighting breaks when trying to Show whitespace and tab characters for C/C++ files (v3.3.1+)

1 participant