feat: correctly parse malformed md syntax

[maximilianfalco]

	Fix CX-2680

🧰 Changes

Created a post process AST-based plugin that runs after remarkParse that fixes and parses malformed MD syntax. We can now handle malformed syntax like

• ** bold**
• __ bold__
• * italic *

Note

The above syntax was previously accepted in the old markdown engine because it used remark-parse version 8. mdxish and mdx uses version 11 which is now built on top of micromark which is significantly more strict, hence why these invalid md syntaxes are not allowed.

Before	After

🧬 QA & Testing

1. Type in malformed MD syntax for bold and italics
2. With the new transformer, we should still be able to render them

• Broken on production.
• Working in this PR app.

[kevinports]

Really nice find that this "regression" is actually come differences in how old/new versions of how remark-parse handles this syntax. I assumed it was something we intentionally added in the legacy markdown package but I couldn't figure out where.

I wish we didn't have to support this because it really isn't valid markdown. But as you've seen with enterprise migrations we need parity with the legacy engine to support customers. So thanks for figuring this out!

Had a handful of comments. Remark plugins aren't my forte so doing my best here.

[maximilianfalco]

@kevinports thanks for the review! I had another look and I forgot to include strikethrough syntaxes using the squiggly "~~" thing. It looks like our editor is already pretty strict about this (ie if there is a space it would not render the strikethrough) but it does fall in the same ballpark as the bold syntax (ie ** and __)

so im just wondering if we should also loop that syntax (ie ~~) in as well or not? cc @rafegoldberg

[kevinports]

so im just wondering if we should also loop that syntax (ie ~~) in as well or not?

Yeah good point. Though I checked on a legacy project and it looks like the legacy engine does not handle malformed strikethroughs like ~~ foo~~. So my vote is to follow suite here and only worry about ** and _. The fewer odd behaviors we have to support the better imo.

[kevinports]

Noticed one more small thing to address before merging. But otherwise looks good to me.

[maximilianfalco]

thanks @kevinports!! idk if we should merge this in right now considering the merge freeze over the holidays but if its alright, I would probably need your help in merging them in... I dont think I have the perms to merge (Im not sure why tbh but Im not seeing any 'merge' buttons on my side)