Add crossgen2 automated SuperPMI collection of Core_Root libraries #47818
Conversation
ghost
added
the
area-CodeGen-coreclr
|
@dotnet/jit-contrib |
|
cc @dotnet/crossgen-contrib |
kunalspathak
left a comment
kunalspathak
left a comment
There was a problem hiding this comment.
Overall looks good (as long as superpmi collection pipeline is green) except a comment about moving --exclude from superpmi.py to superpmi-setup.py.
src/coreclr/scripts/superpmi.py
Outdated
There was a problem hiding this comment.
I would recommend the right way to do this would be to not copy the assemblies that are problematic to the payload directory. superpmi-setup.py already has an ability to do that:
runtime/src/coreclr/scripts/superpmi-setup.py
Line 296 in 4de28f3
Perhaps, just add a condition there that if the collection_name == 'crossgen2', exclude it.
There was a problem hiding this comment.
With that, you won't need any changes in superpmi.py. The script will just act on the assemblies you provided.
There was a problem hiding this comment.
ok, I moved the exclusion to superpmi-setup.py. The superpmi.py -exclude code is not required now, but I'd like to keep it as it could be generally useful.
|
I have kicked off a superpmi collection run based on this PR - https://dev.azure.com/dnceng/internal/_build/results?buildId=978167&view=results |
Never mind. I just saw that it depends on #47779. I have cancelled the run. |
Exclude R2RTest\Microsoft.Build.dll from automated collection. Add a superpmi.py collect `-exclude` option to avoid collecting assemblies known to fail (and cause pop-up failures: see dotnet#47552). (This isn't used by the automated collection, but is generally useful.) Fixes dotnet#47541
BruceForstall
force-pushed
the
AddCrossgen2Collection2
branch
from
f0a4662 to
086fe9f
Compare
Add .dotnet directory to path, so it can be found by superpmi.py when setting up a crossgen2 collection. Extra: if `-temp_dir` is specified to superpmi.py, normalize it to an absolute path.
Fix Windows PATH setting XML separator escape character Add some more logging output
1. exclude more native files from superpmi setup 2. write stdout and stderr files to collection log on failure 3. write superpmi collection error return code to log on failure 4. fix creation of log file directories when `-log_file` argument specified 5. stop logging too-verbose message about CORINFO_SIG_INST handle array illegal values, since it's "by design" for crossgen2
This makes it easier to see the log output without needing to find and download the logfile separately from the CI.
Allow copying .json files, which is required by crossgen2. Don't filter anything in copy of .dotnet directory; I don't know what kind of files are there, and don't want to filter any of them.
|
@kunalspathak I've made a bunch of additional changes here (you've probably noticed), in case you want to review them. |
kunalspathak
left a comment
kunalspathak
left a comment
There was a problem hiding this comment.
LGTM, with a minor suggestion.
| # Need to accept files without any extension, which is how executable filesnames look. | ||
| acceptable_copy = lambda path: (os.path.basename(path).find(".") == -1) or any(path.endswith(extension) for extension in [".py", ".dll", ".so", ".json"]) | ||
|
|
||
| print("is_windows {}".format(is_windows)) |
| "clretwrc.dll", | ||
| "clrgc.dll", | ||
| "clrjit.dll", | ||
| "clrjit_unix_arm_arm.dll", |
There was a problem hiding this comment.
Thanks for doing this. Wanted to do from long time.
They crash during collection and need to be investigated.
ghost
locked as resolved and limited conversation to collaborators
2 participants
Add a superpmi.py collect
-excludeoption to avoid collectingassemblies known to fail (and cause pop-up failures: see #47552).
Use
-excludeto exclude R2RTest\Microsoft.Build.dll.Depends on #47779
Fixes #47541