Changes to CoreCLR workflow

[jkoritzinsky]

PR #31701 is changing the developer workflow for CoreCLR to help increase reliability, reduce local build times, and move us closer to a place where we can better parallelize CI to improve build times.

The largest part of the change is that the build.cmd and build.sh scripts under src/coreclr will be deprecated and later (probably in a few weeks) removed. The src/coreclr/src/build.proj file will also become unused and will be deleted. To build the coreclr product, the supported path will be to build from the root build script.

For those who strongly dislike building with MSBuild, the native runtime build will still be buildable with new build-runtime.cmd and build-runtime.sh scripts in src/coreclr that only build the native runtime build and the cross-arch native build. There will also be separate scripts available for running crossgen on System.Private.CoreLib. All managed builds will be driven exclusively through the Arcade scripting.

The way to control what parts of the CoreCLR build you run will be via the "subsets" infrastructure. See #31701 for specific subset names.

Below are some examples of commands for common workflows:

• Build CoreCLR runtime, System.Private.CoreLib, crossgen System.Private.CoreLib all in Debug. Build all of Libraries in Release. (Common workflow when syncing)

./build -subsetcategory coreclr-libraries -subset runtime-corelib-nativecorelib-all -c Release -runtimeconfiguration Debug

• Build CoreCLR runtime

./build -subsetcategory coreclr -subset runtime

• Build CoreCLR runtime (build-runtime version)

src/coreclr/build-runtime

• Build CoreCLR runtime and Crossgen2

./build -subsetcategory coreclr -subset runtime-tools

The subsets should give as much fine-grained control as was previously provided within the managed components. The native build scripts continue to have some finer-grained control and the build-runtime scripts are available for those who need it.

[stephentoub]

@jkoritzinsky, I'm struggling to follow the examples a little. If today I do this:

src\coreclr\build -release -skiptests

what does that translate to now? Thanks.

[jkoritzinsky]

@jkoritzinsky, I'm struggling to follow the examples a little. If today I do this:

src\coreclr\build -release -skiptests

what does that translate to now? Thanks.

That translates to build -subsetcategory coreclr -c Release.

CoreCLR tests are now only built through the src/coreclr/build-test scripts, not the product build scripts, so there’s no need for a skiptests argument.

[jkotas]

What is going to be equivalent of src\coreclr\build.cmd linuxmscorlib ?

[jkoritzinsky]

What is going to be equivalent of src\coreclr\build.cmd linuxmscorlib ?

The equivalent is ./build -subset corelib -os Linux.

[jkotas]

build -subsetcategory coreclr -c Release

This fails very early for me with:

Restore completed in 78.15 ms for D:\runtime\src\coreclr\src\tools\crossgen2\crossgen2\crossgen2.csproj. 'py' is not recognized as an internal or external command, operable program or batch file. The filename, directory name, or volume label syntax is incorrect. D:\runtime\src\coreclr\src\System.Private.CoreLib\System.Private.CoreLib.csproj(467,5): error MSB3073: The command ""'py' is not recognized as an internal or external command,;operable program or batch file.;C:\Users\jkotas\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\python.exe" -B -Wall "D:\runtime\src\coreclr\/src/scripts/genRuntimeEventSources.py" --man "D:\runtime\src\coreclr\/src/vm/ClrEtwAll.man" --intermediate "D:\runtime\artifacts\obj\coreclr\Eventing\x64\Release"" exited with code 123.

What is the recommended way to deal with the python dependency?

[jkoritzinsky]

That is a bug in eng/python.targets that didn't show up in testing somehow. I'll go fix that today. The workaround is to set the PYTHON environment variable to point to your python installation.

[jkotas]

Thanks! FWIW, I have the python setup according to https://github.com/dotnet/runtime/blob/master/docs/workflow/requirements/windows-requirements.md#python