Skip comments entirely when evaluating indentation

[aiday-mar]

In relation to #209519

As part of the indentation improvement, we would like to entirely strip comments from the current line being considered before evaluating the indentation rules

Simplify the indentation regexes in order not to take into account comment placements for indentation.

[RedCMD]

I think this is a good idea
but would it be optional or forced?

[aiday-mar]

After discussion with my colleagues, we decided this would be forced. That is comments will never be considered in the indentation evaluation.

[aiday-mar]

Fixed with #210641. This PR actually treats strings, regexe and comment tokens in a similar manner. The language configuration brackets in these tokens are stripped and the indentation rules are evaluated on the stripped lines