`Regex.Match` with unexpected result when using `startat` parameter

[lauxjpn]

I just executed the following lines and expected both Regex.Match() calls to be semantically equal.
However, they are not. While the first one works as expected, the second one fails:

const string fooString = "foo BAR foo";
var regex = new Regex(@"^\w+");

var regexSubstringMatch = regex.Match(fooString.Substring(4)); // regexSubstringMatch.Success == true
var regexStartAtMatch = regex.Match(fooString, startat: 4);    // regexStartAtMatch.Success == false

I consider this behavior unexpected.
Here is a full console project for a quick repro:

using System.Diagnostics;
using System.Text.RegularExpressions;

namespace RegexMatchStartAtIssue
{
    internal static class Program
    {
        private static void Main(string[] args)
        {
            const string fooString = "foo BAR foo";
            var regex = new Regex(@"^\w+");

            var regexSubstringMatch = regex.Match(fooString.Substring(4));
            var regexStartAtMatch = regex.Match(fooString, startat: 4);

            Debug.Assert(regexSubstringMatch.Success);
            Debug.Assert(regexSubstringMatch.Value == "BAR");
            Debug.Assert(regexStartAtMatch.Success); // == false
            Debug.Assert(regexStartAtMatch.Value == "BAR");
        }
    }
}

---

Tested on .NET 5.0.100-rc.2.20479.15.

Tagging subscribers to this area: @eerhardt, @pgovind, @jeffhandley
See info in area-owners.md if you want to be subscribed.

Details

---

Issue meta data

Issue content:	I just executed the following lines and expected both `Regex.Match()` calls to be semantically equal. However, they are not. While the first one works as expected, the second one fails: const string fooString = "foo BAR foo"; var regex = new Regex(@"^\w+"); var regexSubstringMatch = regex.Match(fooString.Substring(4)); // regexSubstringMatch.Success == true var regexStartAtMatch = regex.Match(fooString, startat: 4); // regexStartAtMatch.Success == false I consider this behavior unexpected. Here is a full console project for a quick repro: using System.Diagnostics; using System.Text.RegularExpressions; namespace RegexMatchStartAtIssue { internal static class Program { private static void Main(string[] args) { const string fooString = "foo BAR foo"; var regex = new Regex(@"^\w+"); var regexSubstringMatch = regex.Match(fooString.Substring(4)); var regexStartAtMatch = regex.Match(fooString, startat: 4); Debug.Assert(regexSubstringMatch.Success); Debug.Assert(regexSubstringMatch.Value == "BAR"); Debug.Assert(regexStartAtMatch.Success); // == false Debug.Assert(regexStartAtMatch.Value == "BAR"); } } } Tested on .NET 5.0.100-rc.2.20479.15.
Issue author:	lauxjpn
Assignees:	-
Milestone:	-

[danmoseley]

I thought this might be #42409, but this pattern doesn't start with .*. Also, it was fixed in RC2.

[lauxjpn]

I can recheck this tomorrow after the .NET 5 GA release, to verify that this is still an issue (or not) and report back.

[danmoseley]

Ah, it's because your pattern starts with ^. Per the docs,

^ | By default, the match must start at the beginning of the string; in multiline mode, it must start at the beginning of the line.

If you pass a startat that's not zero, it doesn't see the beginning of the string; the ^ will never match, at least unless you have multiple lines and enable multiline matching.

Instead you need \G, per the docs for the startat overload:

If you want to restrict a match so that it begins at a particular character position in the string and the regular expression engine does not scan the remainder of the string for a match, anchor the regular expression with a \G (at the left for a left-to-right pattern, or at the right for a right-to-left pattern). This restricts the match so it must start exactly at startat.

If you change the code to var regex = new Regex(@"\G\w+"); it works as you expect. This assumes you need it to start "on the left" -- otherwise you don't need either ^ or \G

[danmoseley]

Thanks for the report though!

[danmoseley]

If you like, you could click the edit button (top right) on the topic for these overloads and perhaps point out that ^ doesn't "move" with startat.

[lauxjpn]

Instead you need \G, per the docs for the startat overload

Thanks, I think I would not have discovered this via the docs, because this is just such an unexpected behavior to me and the docs are not that expressive about this particular case.

But I guess the intention of the startat parameter then is not to start parsing and evaluating the input from that position, but to return the matches from that position (unless used with \G).
This would mean in my case, that my expression would be evaluated by the Regex engine from the beginning of the string and would result in a single match for the first foo. But since I called it with a startat parameter that points to a position after the first foo, this single match will not be returned.

Since using \G will only work this way for Match() calls with a startat parameter, and would anchor after a previous match for other calls, I guess for my own usage scenarios I will keep using Match(input.Substring(n)) over the startat parameter, so I can use regular expression patterns without having to track and document, with what Match() calls they are supposed to be used.

(I guess the current behavior will not be changed after being this way for 15-20 years or so, but it seems very quirky and unintuitive to me anyway.)

If you like, you could click the edit button (top right) on the topic for these overloads and perhaps point out that ^ doesn't "move" with startat.

I'll do that!

[pgovind]

I think this is a good issue to implement as an Analyzer. At a high level, implementing this as an Analyzer wouldn't take too long and there's obvious value here. I'll look into it and create a PR if it makes sense!

[danmoseley]

Would it be common enough, @pgovind ? It would require (1) non zero startas + (2) pattern beginning with caret + (3) single line mode (the default).

[pgovind]

I don't think it'd be too common, no. I still think it'd be good UX though. I wouldn't prioritize the Analyzer, but the work involved in implementing this will likely be small.