JIT: Use normalized block for comparison in RBO dominator walk

[hez2010]

Fixes an issue where we bail out early in the RBO dominator walk due to comparing a successor that was skipped through empty BBJ_ALWAYS blocks against the current block.

The jit dump was like

Checking BB04 for redundant dominating branches
BB03 and BB04 have shared successor BB06
Optimizing branch in dominating BB03 with relop [000011] based on BB04's relop [000015]

Redundant dominating branch opt in BB03:

removing useless STMT00003 ( 0x008[E--] ... 0x00A )
N004 (  5,  5) [000012] -----------                         ▌  JTRUE     void   $VN.Void
N003 (  3,  3) [000011] -----------                         └──▌  CNS_INT   int    0
 from BB03

BB03 becomes empty
setting likelihood of BB03 -> BB04 from 0.5 to 1

Conditional folded at BB03
BB03 becomes a BBJ_ALWAYS to BB04
Compiler::optRedundantDominatingBranch removed tree:
N004 (  5,  5) [000012] -----------                         ▌  JTRUE     void   $VN.Void
N003 (  3,  3) [000011] -----------                         └──▌  CNS_INT   int    0

continuing to the next immediate dominator
failed -- BB02 is degnerate

after BB03 is folded into an empty BBJ_ALWAYS to BB04, we continue walking to the next dominator BB02, but when checking BB02, we skipped side effect free blocks on its successors so that we end up comparing BB04 against BB03.

Codegen diff:

 G_M29168_IG01:  ;; offset=0x0000
        sub      rsp, 40
                                                 ;; size=4 bbWeight=0.50 PerfScore 0.12
 G_M29168_IG02:  ;; offset=0x0004
-       cmp      ecx, 2
-       jle      SHORT G_M29168_IG03
        cmp      ecx, 8
        jle      SHORT G_M29168_IG03
        mov      ecx, 1
        call     [System.Console:WriteLine(int)]
-                                                ;; size=21 bbWeight=0.50 PerfScore 2.88
+                                                ;; size=16 bbWeight=0.50 PerfScore 2.25
-G_M29168_IG03:  ;; offset=0x0019
+G_M29168_IG03:  ;; offset=0x0014
        nop
                                                 ;; size=1 bbWeight=1 PerfScore 0.25
-G_M29168_IG04:  ;; offset=0x001A
+G_M29168_IG04:  ;; offset=0x0015
        add      rsp, 40
        ret
                                                 ;; size=5 bbWeight=1 PerfScore 1.25

Fixes #127432

/cc: @AndyAyersMS

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

Fixes an RBO dominator-walk edge case in optRedundantDominatingBranch where the walk can bail out after prior folding creates an empty BBJ_ALWAYS chain, by ensuring comparisons are done against the normalized (side-effect-free-skipped) “current” block.

Changes:

• Normalize currentBlock via skipSideEffectFreeBlocks before comparing it against dominator successors in the RBO dominator walk.
• Add a new dominating-branch test case (Dom_05) to exercise the reported control-flow shape.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/coreclr/jit/redundantbranchopts.cpp	Normalizes the block being compared in the dominator walk to avoid mismatches caused by empty BBJ_ALWAYS chains.
src/tests/JIT/opt/RedundantBranch/RedundantBranchDominating.cs	Adds a new nested-dominating-branch scenario (Dom_05) and a corresponding test.

[hez2010]

Diffs

[AndyAyersMS]

Thanks for spotting this.