Skip to content

fix(cli): parse markdown command frontmatter on Windows CRLF/BOM#2078

Merged
tanzhenxin merged 2 commits intoQwenLM:mainfrom
zy6p:fix/2053-markdown-frontmatter-windows
Mar 4, 2026
Merged

fix(cli): parse markdown command frontmatter on Windows CRLF/BOM#2078
tanzhenxin merged 2 commits intoQwenLM:mainfrom
zy6p:fix/2053-markdown-frontmatter-windows

Conversation

@zy6p
Copy link
Copy Markdown
Contributor

@zy6p zy6p commented Mar 4, 2026

Summary

  • normalize markdown command content before frontmatter parsing by stripping UTF-8 BOM and converting CRLF to LF
  • make frontmatter regex robust for empty frontmatter and optional trailing newline
  • add regression tests for CRLF and BOM parsing scenarios
  • add loader-level regression test to ensure markdown command descriptions are read correctly from BOM+CRLF files

Testing

  • npm run test --workspace=packages/cli -- src/services/markdown-command-parser.test.ts src/services/FileCommandLoader-markdown.test.ts

Fixes #2053

@Aayushyaash
Copy link
Copy Markdown
Contributor

@zy6p

This looks like the same class of Windows newline/BOM parsing issue.

There is a related open PR (#1890) that introduces a normalizeContent() helper which strips UTF-8 BOM and normalizes CRLF/CR → LF before parsing. The goal there is to centralize newline normalization so different parsers don’t each implement their own handling.

If it makes sense, this markdown frontmatter parsing could potentially reuse that helper instead of adding another normalization step in the CLI parser. That would keep newline/BOM handling consistent across the repo and avoid duplicated logic.

What do you think?

@zy6p
Copy link
Copy Markdown
Contributor Author

zy6p commented Mar 4, 2026

@Aayushyaash Thanks for the suggestion. I updated this PR to reuse the shared normalizeContent() helper from core utils (instead of parser-local normalization), so BOM/newline handling stays centralized.\n\nAlso added coverage for:\n- core normalizeContent() (BOM, CRLF, CR-only, mixed line endings)\n- CLI markdown frontmatter parsing with CRLF/BOM/CR-only content\n\nLatest commit: 05f2224.

Copy link
Copy Markdown
Collaborator

@tanzhenxin tanzhenxin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@tanzhenxin tanzhenxin self-assigned this Mar 4, 2026
@tanzhenxin tanzhenxin merged commit d9bc66c into QwenLM:main Mar 4, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows: Markdown frontmatter not parsed when file uses CRLF or UTF-8 BOM (description not displayed)

3 participants