[meta] coredistools in jitutils

[echesakov]

#289 will add support for building coredistools from dotnet/jitutils repo. This PR adds multiple scripts for enlisting and building LLVM components as well coredistools itself. At the end of the corresponding pipeline run, each job produces a coredistools library binary file (e.g. coredistools.dll, libcoredistools.dylib) for a specified host platform (e.g. windows-arm64). Such binaries files support all the target architectures, so, presumably, they can be used in cross-architecture scenarios (e.g. running superpmi replay with altjit on windows-x86 host targetting linux-arm with mch files collected on linux-arm).

This issue is to track the remaining work:

• Test all the binaries with SuperPMI for all platforms in the matrix:
  • windows-x86 replay/collect with 260 framework libraries
  • windows-x64 replay/collect with 260 framework libraries
  • windows-arm (native) replay/collect with 260 framework libraries
  • windows-arm (on windows-x86 host) replay with windows-arm *.mch and clrjit_win_arm_x86.dll
  • windows-arm64 (native)
  • windows-arm64 (on windows-x64 host)
  • linux-x64
  • linux-arm (native) blocked by [linux-arm] superpmi replay fails due to illegal alignment runtime#42641
  • linux-arm64 (native)
  • linux-arm64 (on linux-x64 host)
  • (Optional) linux-x64 (on windows-x64 host)
  • (Optional) windows-x64 (on linux-x64 host)
  • (Optional) linux-arm (on windows-x86 host)
  • (Optional) linux-arm64 (on windows-x64 host)
  • (Optional) windows-arm64 (on linux-x64 host)
• Test all the binaries with GCStress for the following platforms:
  • windows-x86
  • windows-x64
  • linux-x64
• Test all the binaries with R2RDump for the following host platforms:
  • windows-x86
  • windows-x64
  • linux-x64
• Fix issue on Arm32 by porting some of the changes in [WIP] Support Arm32 and Arm64 in nearDiff llilc#1098. Update: fixed with CoreDisTools on Arm #291
• Fix issue on Arm64 and propose a solution to Arm64 literal pool issue as discussed in [WIP] Support Arm32 and Arm64 in nearDiff llilc#1098 This should resolve Need coredistools for arm32/arm64 runtime#10281
• Automate publishing (i.e. add support for producing nuget packages and publishing them as pipeline artifacts). Note that publishing the nuget packages to a nuget repository is outside of the scope. It can accomplished after the repo will onboard onto Arcade - I partly finished this task in Onboarding onto Arcade #219
• Document the completed work for the future maintenance purposes
• (Optional) Update to LLVM 10.0.1 that is the current stable version of LLVM. At the moment, coredistools in that PR uses LLVM 9.0.1 and the update to higher version requires changing coredistools.cpp itself since there were some changes in LLVM API between 9 and 10.

[echesakov]

@dotnet/jit-contrib
Also cc @janvorli and @davidwrighton who might be interested in this work

[echesakov]

This could be closed.

All the blocking issues were resolved and the testing was done naturally by us using SuperPMI on day-to-day basis.

The only remaining actionable item was to start using LLVM 10 (well, LLVM 13 by now) but there is no immediate plans to do this and the value is not clear at this point.