Ensure we run the Crossgened test executable in R2R tests#11272
Conversation
|
@dotnet-bot test Windows_NT Release pri1r2r |
|
@dotnet-bot test Ubuntu Release pri1r2r |
|
Tests seem to pass locally, will wait for them to pass in the CI as well. |
|
Just be curious: the CI passed when I submit #10525 . Is this test not being added to CI at that time? |
|
@mazong1123 that's correct, the R2R tests in the background. They only run in PR's if you request them (as I did above). |
|
CI has test failures @wtgodbe for R2R. |
|
Yeah, the Windows job had 1 test fail, which looks like a real issue that should be investigated - as for the Ubuntu job, it failed in trying to precompile ildasmrc.dll - it looks like similar failures have existed for a long time (about a month) - https://ci.dot.net/job/dotnet_coreclr/job/master/job/x64_release_ubuntu_pri1r2r_flow/. I'm guessing we need to either add a few .dll's to the exclusion list at https://github.com/dotnet/coreclr/blob/master/tests/skipCrossGenFiles.x64.txt, or rethink the way we do crossgening of Core_Root on non-windows. |
|
This is the Windows failure: Does it fail for you locally? If it does, based upon the error message, it could be related to the engineering change you are making. Can you confirm, just for this test, if the original binary was not renamed to .org, does the test pass? |
|
Ubuntu Failure is the following: The infrastructure appears to be crossgenning native binary and failing - can you confirm if that is the case? |
|
@gkhanna79 where are you seeing the superpmi-shim-simple.dll failure? I only see ildasmrc.dll failing to precompile. Either way, this lines up with the other Ubuntu R2R failures that have been happening for the last month - we're trying to crossgen native binaries. I'll look into that. As for the Windows failure, I've confirmed the same happens locally, though the directory looks as expected, and the .org file is present after running the .cmd. Also, if I modify the .cmd to use the original extensions from before this change, crossgen happens successfully, and I can rename the .ni.exe to '.exe' and have the test pass. Not sure what's special about that particular test... |
|
Think I found the problem for non-Windows: https://github.com/dotnet/coreclr/blob/master/tests/runtest.sh#L410 The code for COR_E_ASSEMBLYEXPECTED is 0x80131018, which we do see in the error log output for the failed crossgen - I suspect that maybe up until a month ago, we printed 'COR_E_ASSEMBLYEXPECTED' in the logs, but now print the error code instead. I'll see what happens if I change that line to use the hex code instead. |
|
I sent some jobs to the CI with the change I mentioned above, and we no longer fail with trying to precompile the native assemblies from before, but we do get the following failure trying to precompile System.Private.Corelib.dll:
I am, however, able to successfully precompile System.Private.Corelib.dll on my Ubuntu14.04 VM locally. @gkhanna79 any idea what's going on here? |
|
Are you synced to latest when trying this? |
wtgodbe
force-pushed
the
CrossgenMove
branch
3 times, most recently
from
884189f to
7d557fc
Compare
|
@dotnet-bot test Windows_NT Release pri1r2r |
|
@dotnet-bot test Ubuntu Release pri1r2r |
|
@dotnet-bot test Tizen armel Cross Release Build |
Update if statement
|
@dotnet-bot test Ubuntu Release pri1r2r |
|
@dotnet-bot test Windows_NT Release pri1r2r |
Were these passing earlier? How about you merge the PR and look into if this failure persists? |
|
I haven't seen those failures before, and the build history for r2r jobs doesn't go back far enough to see if they were failing before the infra issue popped up. I'll merge this & keep an eye on mission control. |
dotnet#10525 regressed loading ni.exe files - after this change, the runtime no longer looks for the .ni.exe variants of .exe files because the .exe is considered trusted. This regression was worked around in dotnet#11272 for crossgen testing, but we still have tests (such as tests\src\readytorun\tests) that are now not testing anything because we're silently falling back to JIT. This makes corerun add both .exe and .ni.exe into the trusted assembly list.
Reverts dotnet/coreclr#10525. This broke probing for .ni.exe images. If we run `corerun foo.exe` for something that has a `foo.ni.exe`, we wouldn't pick up the native image because foo.exe is considered trusted and foo.ni.exe is not. This means we would fall back to JIT. But only if foo.exe is not in CORE_ROOT. If it is, it works... The pull request broke crossgen testing that was eventually worked around in dotnet/coreclr#11272, but it's also breaking the tests in tests\src\readytorun\tests (we just didn't find out because they're falling back to JIT). We could work around there too, but this took half a day for me to investigate, so I don't want to leave the landmine there for someone else to lose a leg over (the fact that ni probing works if you put the exe and ni.exe into CORE_ROOT was a major timesink when troubleshooting). I don't see much point in this change. It's fixing the case when someone copied something.exe to CORE_ROOT and then tries to `corerun some\other\version\of\something.exe`, but it doesn't fix the situation when the something.exe depends on dependency.dll - we would still pick up dependency.dll from CORE_ROOT, which is likely the wrong version. The only way to win this game is not to play it - don't put stuff in CORE_ROOT.
…" (#31848) Reverts dotnet/coreclr#10525. This broke probing for .ni.exe images. If we run `corerun foo.exe` for something that has a `foo.ni.exe`, we wouldn't pick up the native image because foo.exe is considered trusted and foo.ni.exe is not. This means we would fall back to JIT. But only if foo.exe is not in CORE_ROOT. If it is, it works... The pull request broke crossgen testing that was eventually worked around in dotnet/coreclr#11272, but it's also breaking the tests in tests\src\readytorun\tests (we just didn't find out because they're falling back to JIT). We could work around there too, but this took half a day for me to investigate, so I don't want to leave the landmine there for someone else to lose a leg over (the fact that ni probing works if you put the exe and ni.exe into CORE_ROOT was a major timesink when troubleshooting). I don't see much point in this change. It's fixing the case when someone copied something.exe to CORE_ROOT and then tries to `corerun some\other\version\of\something.exe`, but it doesn't fix the situation when the something.exe depends on dependency.dll - we would still pick up dependency.dll from CORE_ROOT, which is likely the wrong version. The only way to win this game is not to play it - don't put stuff in CORE_ROOT.
5 participants
CC @RussKeldorph @gkhanna79
This will fix the Crossgen test failures introduced by #10525. Have tested that a test .cmd with the expected outcome will pass locally, currently in the process of testing building the branch from scratch & running tests.