Rework RegexGenerator incremental source model to support value equality

[eiriktsarpalis]

Fixes #125409

Discovered while working on #125404 — the RegexGenerator's incremental source model lacks proper value equality, causing the source output callback to fire on every compilation change regardless of whether regex patterns have actually changed.

Approach

Refactors the generator following the STJ/Configuration Binder pattern with distinct Parser/Emitter files and deeply equatable model types (*Spec records using ImmutableEquatableArray, ImmutableEquatableSet, and ImmutableEquatableDictionary).

The emitter re-parses the regex from Pattern + Options + CultureName during emission to reconstruct the mutable RegexTree/RegexNode objects rather than porting the 5800-line emitter to use *Spec types. This re-parse is fast (~1ms) and only runs on cache misses, while the expensive emission (10-100ms) is fully skipped on cache hits.

19 new incremental compilation tests verify caching behavior via TrackedOutputSteps.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

This pull request refactors the RegexGenerator incremental source generator pipeline to use deeply equatable model types, enabling Roslyn incremental caching to skip expensive emission when regex inputs haven’t changed.

Changes:

• Replaced the heterogeneous ImmutableArray<object> incremental model with equatable *Spec model records (using ImmutableEquatableArray / new ImmutableEquatableDictionary).
• Reworked the pipeline to split source emission from diagnostic reporting and re-parse the regex during emission only on cache misses.
• Added a new incremental-caching-focused test suite and wired it into the functional test project.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/System.Text.RegularExpressions.Tests.csproj	Adds the new incremental generator tests to the test project.
src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexGeneratorIncrementalTests.cs	New tests validating incremental caching behavior via tracked generator steps.
src/libraries/System.Text.RegularExpressions/gen/System.Text.RegularExpressions.Generator.csproj	Links shared equatable collection helpers and required-member polyfills into the generator project.
src/libraries/System.Text.RegularExpressions/gen/RegexGenerator.cs	Pipeline rewrite to produce a structurally equatable model and re-parse on emission.
src/libraries/System.Text.RegularExpressions/gen/RegexGenerator.Conversion.cs	Converts RegexTree / analysis into equatable *Spec snapshots.
src/libraries/System.Text.RegularExpressions/gen/Model/RegexGenerationSpec.cs	New top-level equatable incremental cache boundary model.
src/libraries/System.Text.RegularExpressions/gen/Model/RegexMethodSpec.cs	New per-member equatable model for generation inputs.
src/libraries/System.Text.RegularExpressions/gen/Model/RegexTreeSpec.cs	New equatable representation of regex trees and baked-in analysis flags.
src/libraries/System.Text.RegularExpressions/gen/Model/RegexNodeSpec.cs	New equatable representation of regex parse nodes.
src/libraries/System.Text.RegularExpressions/gen/Model/RegexTypeSpec.cs	New equatable representation of declaring type hierarchy.
src/libraries/System.Text.RegularExpressions/gen/Model/FindOptimizationsSpec.cs	New equatable representation of find-optimization state.
src/libraries/System.Text.RegularExpressions/gen/Model/FixedDistanceSetSpec.cs	New equatable representation of fixed-distance set optimization data.
src/libraries/Common/src/SourceGenerators/ImmutableEquatableDictionary.cs	New shared equatable dictionary helper for structural equality in generator models.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.Text.RegularExpressions/gen/RegexGenerator.Parser.cs:150

• The spec currently doesn’t preserve the cultureName argument from [GeneratedRegex]. CreateRegexTreeSpec later relies on tree.Culture?.Name, but RegexParser.Parse sets RegexTree.Culture to non-null only for ignore-case backreferences; for most ignore-case regexes the tree culture will be null. That means the emitter may re-parse using InvariantCulture even when a culture was specified, potentially generating incorrect code for culture-sensitive case folding. Store the requested/selected culture name directly on RegexMethodSpec (or alongside Tree) and use that for emission and de-duping.

            string? pattern = items[0].Value as string;
            int? options = null;
            int? matchTimeout = null;
            string? cultureName = string.Empty;
            if (items.Length >= 2)
            {
                options = items[1].Value as int?;
                if (items.Length == 4)
                {
                    matchTimeout = items[2].Value as int?;
                    cultureName = items[3].Value as string;
                }
                // If there are 3 parameters, we need to check if the third argument is
                // int matchTimeoutMilliseconds, or string cultureName.
                else if (items.Length == 3)
                {
                    if (items[2].Type?.SpecialType == SpecialType.System_Int32)
                    {
                        matchTimeout = items[2].Value as int?;
                    }
                    else
                    {
                        cultureName = items[2].Value as string;
                    }

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[eiriktsarpalis]

FYI @ericstj @tarekgh

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

---

You can also share your feedback on Copilot code review. Take the survey.