Skip to content

Fix inline highlighter quote handling for Shell and JS/TS nested expressions#134

Merged
Stasshe merged 5 commits intomainfrom
copilot/fix-inline-highlighter-quote-handling
Dec 3, 2025
Merged

Fix inline highlighter quote handling for Shell and JS/TS nested expressions#134
Stasshe merged 5 commits intomainfrom
copilot/fix-inline-highlighter-quote-handling

Conversation

Copy link
Contributor

Copilot AI commented Dec 3, 2025

Shell strings containing $() or ${} and JS/TS template literals with nested ${} caused "phase reversal" - content inside and outside quotes was incorrectly swapped. The regex-based tokenizer couldn't handle nested structures.

# Before: tokenized as ["][hello ][$(whoami)][ world]["]
# After:  tokenized as ["hello $(whoami) world"]
echo "hello $(whoami) world"
// Before: tokenized as [`Nested ${`][inner][`}`]
// After:  tokenized as [`Nested ${`inner`}`]
const x = `Nested ${`inner`}`

Changes

  • Shell/Bash (bash, sh, zsh, fish):

    • parseShellDoubleQuotedString: Handles $(), ${}, backticks, escapes within double-quoted strings
    • parseShellCommandSubstitution: Handles nested parens and quotes inside $()
    • tokenizeShell: State-based tokenizer replacing regex for strings
  • JavaScript/TypeScript (js, ts, jsx, tsx, etc.):

    • parseJsTemplateString: Handles ${} interpolation in template literals
    • parseJsTemplateExpression: Handles nested templates, strings, braces inside ${}
    • tokenizeJsTs: State-based tokenizer for template literals

The tokenize() function now dispatches to language-specific tokenizers based on the language parameter.

Original prompt

inlinedhighlighterの、"や'の扱いが雑。ASTは使いたくないが、もうちょいちゃんとやって。bashとかの、$()や、閉じるとき、"が認識されず、フェーズが移らず、その後、"の中身と外が逆転する。この逆転現象が起きないようになれば、ある程度のミスもまぁいい。できるだけ正確に。


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: Stasshe <133995608+Stasshe@users.noreply.github.com>
Co-authored-by: Stasshe <133995608+Stasshe@users.noreply.github.com>
Co-authored-by: Stasshe <133995608+Stasshe@users.noreply.github.com>
Co-authored-by: Stasshe <133995608+Stasshe@users.noreply.github.com>
@Stasshe Stasshe temporarily deployed to copilot/fix-inline-highlighter-quote-handling - pyxis-code PR #134 December 3, 2025 07:29 — with Render Destroyed
Copilot AI changed the title [WIP] Fix inline highlighter quote handling issues Fix inline highlighter quote handling for Shell and JS/TS nested expressions Dec 3, 2025
Copilot AI requested a review from Stasshe December 3, 2025 07:35
@Stasshe Stasshe marked this pull request as ready for review December 3, 2025 07:37
@Stasshe Stasshe merged commit 698fa68 into main Dec 3, 2025
@Stasshe Stasshe deleted the copilot/fix-inline-highlighter-quote-handling branch December 3, 2025 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants