JIT: Fix uninitialized gtCallMethHnd for indirect calls causing false recursive tail call detection#127293
Merged
AndyAyersMS merged 3 commits intodotnet:mainfrom Apr 24, 2026
Conversation
… recursive tail call detection PR dotnet#125211 broke the union between gtCallMethHnd and gtCallAddr into separate fields but did not initialize gtCallMethHnd for CT_INDIRECT calls. Since the arena allocator does not zero memory in Release builds, gtCallMethHnd contains stale data that can accidentally match the compiled method's handle, causing gtIsRecursiveCall() to return true. Combined with the CORINFO_VIRTUALCALL_LDVIRTFTN import path (used for generic interface calls in R2R) which creates CT_INDIRECT calls without GTF_CALL_VIRT_* flags, the recursive tail call optimization incorrectly transforms the call into a backward jump, creating an infinite allocating loop. Fix by: 1. Initializing gtCallMethHnd = NO_METHOD_HANDLE in gtNewCallNode for CT_INDIRECT 2. Adding a defense-in-depth check in gtIsRecursiveCall to return false for indirect calls 3. Initializing gtCallMethHnd = NO_METHOD_HANDLE in gtCloneExprCallHelper for CT_INDIRECT Fixes dotnet#126930 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
the
area-CodeGen-coreclr
Contributor
|
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a CoreCLR JIT correctness issue introduced when GenTreeCall::gtCallMethHnd was split from the former gtCallAddr union: indirect call nodes (CT_INDIRECT) could leave gtCallMethHnd uninitialized, leading to nondeterministic false “recursive tail call” detection and potential infinite loops/OOMs.
Changes:
- Initialize
gtCallMethHndtoNO_METHOD_HANDLEfor newly createdCT_INDIRECTcalls. - Initialize
gtCallMethHndtoNO_METHOD_HANDLEwhen cloningCT_INDIRECTcalls. - Add a guard in
gtIsRecursiveCall(GenTreeCall*)to always returnfalseforCT_INDIRECTcalls (defense-in-depth).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/coreclr/jit/gentree.cpp |
Ensures indirect call creation/cloning does not leave gtCallMethHnd holding stale arena data. |
src/coreclr/jit/compiler.h |
Prevents recursive-call checks from consulting method handles on indirect calls. |
This was referenced Apr 23, 2026
Open
Member
Author
|
@dotnet/jit-contrib PTAL Suggest we take this to fix the ci-optional issue and then circle back to add better protection. |
jakobbotsch
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was generated with the assistance of GitHub Copilot.
Fixes #126930
Problem
PR #125211 removed the union between
gtCallMethHndandgtCallAddrinGenTreeCall, making them separate fields. However,gtNewCallNodewas not updated to initializegtCallMethHndforCT_INDIRECTcalls. Since the JIT arena allocator doesn't zero memory in Release builds,gtCallMethHndcontains stale arena data.When the stale data coincidentally matches the compiled method's handle,
gtIsRecursiveCall()falsely returns true for indirect calls. Combined with!call->IsVirtual()being true forCORINFO_VIRTUALCALL_LDVIRTFTNcalls (which lackGTF_CALL_VIRT_*flags),fgMorphRecursiveFastTailCallIntoLoopincorrectly transforms the call into a backward jump — creating an infinite allocating loop.This only manifests non-deterministically depending on arena memory layout, which is why it appears on x64 CI but not on arm64 or in local builds.
Fix
Three surgical changes:
gentree.cpp(gtNewCallNode): InitializegtCallMethHnd = NO_METHOD_HANDLEforCT_INDIRECTcalls — the primary fix that prevents stale data.compiler.h(gtIsRecursiveCall): Defense-in-depth — returnfalseforCT_INDIRECTcalls since indirect calls can never be recursive by definition.gentree.cpp(gtCloneExprCallHelper): Same initialization for cloned indirect calls to prevent the same issue in clone paths.