Investigation: IJW HRException with 17+ by-ref parameters on x64

[Copilot]

Note

This content was generated by GitHub Copilot.

Calling a C++/CLI managed static with 17+ const T& parameters from native code throws HRException on .NET 9/10 (worked on .NET 8). Reproduction requires Windows + MSVC + IJW, which is not available in this environment, so no code change is being proposed — this PR is an investigation report with a pointer to the likely regression.

Description

• No source changes. Investigation-only PR.
• Likely regression: PR Inject IJW Copy Constructor calls in the JIT instead of in the IL stubs #106000 / Inject IJW Copy Constructor calls in the JIT instead of in the IL stubs #106424 (commit cfa0cc5b0282, "Inject IJW Copy Constructor calls in the JIT instead of in the IL stubs"), which re-enabled GS (buffer security) checks on Reverse P/Invoke IL stubs.
• Likely site: Compiler::gsCopyIntoShadow in src/coreclr/jit/gschecks.cpp. There is a TARGET_X86 && FEATURE_IJW branch that, for opts.IsReversePInvoke(), inserts the shadow copy after CORINFO_HELP_JIT_REVERSE_PINVOKE_ENTER (i.e. after the GC transition). The fall-through path used on x64 inserts the copy at the start of fgFirstBB — before the transition:

// non-x86/IJW fall-through, inserts BEFORE the reverse-PInvoke transition:
GenTree* src   = gtNewLclvNode(lclNum, varDsc->TypeGet());
GenTree* store = gtNewStoreLclVarNode(shadowLclNum, src);
LIR::AsRange(fgFirstBB).InsertAtBeginning(src, store);

• Hypothesis: with 18 byref params, GS marks all of them as vulnerable pointers (lvIsPtr = 1) and emits a shadow copy per param. The "17" threshold is probably where cumulative frame / stack-arg load first surfaces a latent ordering bug against the P/Invoke-enter helper. This is unverified.
• Ruled out (by reading, not by running):
  • ijwhost start_runtime_thunk_stub and TheUMEntryPrestub prologues — both preserve the caller's stack-arg region across register/XMM save+restore and TAILJMP_RAX. Alignment and shadow-space math check out for >16 args.
  • patch_vtable_entries / FixupVTables / UMEntryThunk — no arg-count-dependent code paths.
• Existing IJW tests (src/tests/Interop/IJW/**) do not cover managed static vtfixup entries with ≥17 by-ref parameters, which is consistent with the regression slipping through.

Suggested next steps (for a reviewer with a Windows box)

• Run the repro with DOTNET_JitDump=IL_STUB* / DOTNET_JitDisasm=IL_STUB* and confirm whether shadow copies for the 18 params are emitted before JIT_REVERSE_PINVOKE_ENTER.
• If confirmed, generalize the TARGET_X86 && FEATURE_IJW branch in gsCopyIntoShadow (and the matching branch in gsParamsToShadows) to fire for all opts.IsReversePInvoke() targets.
• Add a regression test under src/tests/Interop/IJW/ covering a C++/CLI static exposed via vtfixup with ≥17 const T& parameters.

cc @AaronRobinsonMSFT @jkoritzinsky.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[jkotas]

Superseded by #127182