build -help needs to be more helpful

[BruceForstall]

Running build -help displays help for building the repo, but it needs to be improved. Currently, it displays:

Common settings:
  -subset                   Build a subset, print available subsets with -subset help
  -subsetCategory           Build a subsetCategory, print available subsetCategories with -subset help
  -vs                       Open the solution with VS for Test Explorer support. Path or solution name (ie -vs Microsoft.CSharp)
  -os                       Build operating system: Windows_NT or Unix
  -arch                     Build platform: x86, x64, arm or arm64
  -configuration            Build configuration: Debug, Release or [CoreCLR]Checked (short: -c)
  -runtimeConfiguration     Runtime build configuration: Debug, Release or [CoreCLR]Checked
  -librariesConfiguration   Libraries build configuration: Debug or Release
  -verbosity                MSBuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic] (short: -v)
  -binaryLog                Output binary log (short: -bl)
  -help                     Print help and exit (short: -h)

Actions (defaults to -restore -build):
  -restore                Restore dependencies (short: -r)
  -build                  Build all source projects (short: -b)
  -buildtests             Build all test projects
  -rebuild                Rebuild all source projects
  -test                   Build and run tests (short: -t)
  -pack                   Package build outputs into NuGet packages
  -sign                   Sign build outputs
  -publish                Publish artifacts (e.g. symbols)
  -clean                  Clean the solution

Libraries settings:
  -framework              Build framework: netcoreapp5.0 or net472 (short: -f)
  -coverage               Collect code coverage when testing
  -testscope              Scope tests, allowed values: innerloop, outerloop, all
  -allconfigurations      Build packages for all build configurations

Command-line arguments not listed above are passed thru to msbuild.
The above arguments can be shortened as much as to be unambiguous (e.g. -con for configuration, -t for test, etc.).

Suggestions:

1. Options should state what the default is (what would be used if nothing is specified on this machine, in this environment). E.g., OS, arch/configuration, runtimeConfiguration, librariesConfiguration, etc.
2. There should be examples at the end with various suggested command-lines for common usages.
3. There should be a better description of what a "subset" is and what a "subsetCategory" is, and how they differ.
4. Also, can you specify one subsetCategory with one subset, and another subsetCategory with a different subset?
5. nit: the option list should be better ordered. E.g., start with "-help", then "-subsetCategory" (next most important?), then subset, then os/arch/configuration, etc.
6. nit: in "... passed thru to ...", replace "thru" with "through".

When I run build -subset help, it shows:

  Accepted Subset values:
  - RegenerateReadmeTable [only runs on demand]
      Regenerates the table of asset links in the README.md file.
  - Libraries|All
      The .NET libraries comprising the shared framework.
  - CoreClr|Runtime
      The CoreCLR .NET runtime.
  - CoreClr|LinuxDac
      The cross-OS Windows->libc-based Linux DAC. Skipped on x86.
  - CoreClr|AlpineDac [only runs on demand]
      The cross-OS Windows->musl-libc-based Linux DAC. Skipped on x86.
  - CoreClr|CoreLib
      The managed System.Private.CoreLib library for CoreCLR.
  - CoreClr|NativeCoreLib
      Run crossgen on System.Private.CoreLib library for CoreCLR.
  - CoreClr|Tools
      Managed tools that support CoreCLR development and testing.
  - CoreClr|Packages
      The projects that produce NuGet packages for the CoreCLR runtime, crossgen, and IL tools.
  - Mono|All
      The Mono .NET runtime.
  - Installer|All
      The .NET Core hosts, hosting libraries, bundles, and installers. Includes these projects' tests.
  - Installer|CoreHost
      The .NET Core hosts.
  - Installer|Managed
      The managed .NET hosting projects. This includes PlatformAbstractions, DependencyModel, and HostModel.
  - Installer|DepProj
      The dependency projects. These gather shared framework files and run crossgen on them to turn them into ready-to-run (R2R) assemblies for the current platform.
  - Installer|PkgProj
      The packaging projects. These produce NETCoreApp assets: NuGet packages, installers, zips, and Linux packages.
  - Installer|Bundle
      The shared framework bundle installer projects. Produces .exe installers for Windows.
  - Installer|Installers
      Generates additional installers. This produces the shared frameworks and their installers.
  - Installer|Test
      The test projects. Note that building this doesn't execute tests: you must also pass the '-test' argument.

F:\gh\runtime2\eng\Build.props(53,5): error : Choose a subset to use, or do not specify a subset to perform the full build. [F:\NugetCache\microsoft.dotnet.arcade.sdk\5.0.0-beta.20153.1\tools\Build.proj]

Build FAILED.

F:\gh\runtime2\eng\Build.props(53,5): error : Choose a subset to use, or do not specify a subset to perform the full build. [F:\NugetCache\microsoft.dotnet.arcade.sdk\5.0.0-beta.20153.1\tools\Build.proj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.38
(NETCORE_ENGINEERING_TELEMETRY=Category) Build failed.

Suggestions:

1. I'm asking for help. It shouldn't display all the "error" / "Build FAILED" / "Warning(s)" / "Error(s)" / "Time Elapsed" / telemetry stuff. And the build shouldn't be considered failed!
2. I don't understand the syntax. What does "CoreClr|Packages" mean? Is CoreClr|Packages the name of a subset? Or is Packages the name of a subset only applicable to CoreClr? If the latter, is CoreClr the subsetCategory?
3. Are the subsets and subsetCategories case sensitive?
4. Changes to CoreCLR workflow #32991 seems to indicate that you can specify multiple subsets by concatenating them with dashes in between (e.g., runtime-tools). This is quite weird, and not described anywhere. I would generally expect to be able to do -subset runtime tools or -subset runtime -subset tools instead.

Finally, CoreClr should be consistently cased CoreCLR.

[BruceForstall]

@jkoritzinsky @dotnet/runtime-infrastructure

[ViktorHofer]

I agree with your suggestions. We should also improve the subset system and try to merge the subset and subsetcategory parameters which were necessary back when we merge three different repos together but are less important now as we consolidate more code.

[danmoseley]

As discussed offline in Teams, my impression is that it may be better to make the system less complex (eg., perhaps combine the subset and subsetCategory concepts) rather than paper over the complexity with online help and documentation. If it's confusing several members of our own team, it's likely to be causing even more confusion to our contributors.

[ivdiazsa]

Several points have been improved, such as removing subsetCategory and changing the pipe notation to a dot. However, there are other points that could be improved upon:

• Add the default values to the help (OS, Arch, Configurations).
• Sort the flags alphabetically.
• Remove the error when passing -subset help.

I can work on these improvements and extend the first two to build.sh as well. I have an observation though, regarding the last point. The subset help is printed directly by MSBuild and its command built in build.ps1 needs to include the -build flag. Otherwise, it just shows a Build succeeded message but does not print the subsets' descriptions. I could do away with the error message, but I'm not sure if it's possible to remove the "Elapsed Time" metrics.

@BruceForstall @danmosemsft

[safern]

but I'm not sure if it's possible to remove the "Elapsed Time" metrics.

I don't think so because -subset help is generated from MSBuild. Maybe there is a flag to tell MSBuild to not print these metrics?

[danmoseley]

You can suppress "Build Succeeded", the warning/errors count, and the elapsed time by passing "NOSUMMARY" eg /clp:nosummary

You can't suppress any of those individually:
https://github.com/microsoft/msbuild/blob/b0d72851842af6f6b50ad62e6e2077790ee5de29/src/Build/Logging/ParallelLogger/ParallelConsoleLogger.cs#L266-L325