Merge various restore tasks into one Sync target

[weshaggard]

There were various restore targets happening independently
when they should all be combined so the sync step can be
independent from the build step. This change merges them
together under the Sync target.

In particular this moves RestoreOptData and RestoreNETCorePlatforms
to be part of the sync step instead of being individually ran.

Should fix https://github.com/dotnet/coreclr/issues/17176.
Follow-up on #17136.

cc @dpodder @wtgodbe

[weshaggard]

Turns out CI does things differently then official builds (:(). Official builds call sync first but CI and dev builds do not so they will not restore the packages before the native build which needs the PGO data. So I'm going to still keep the optdata restore in the scripts on top of making it get restored during sync. That should enable both scenarios to work.

FYI @RussKeldorph @jashook

[wtgodbe]

LGTM

[weshaggard]

test Ubuntu x64 Formatting

[weshaggard]

@BruceForstall any idea what is going on in the formatting CI leg?

Formatting jit directory.
11:18:50 Can't find compile_commands.json file. Running configure.
11:18:51 There was an error running CMake to generate compile_commands.json. Please run build.sh configureonly

[BruceForstall]

I'm not sure how the formatting jobs internals work. @adiaaida can you advise?

[weshaggard]

test Ubuntu x64 Formatting

[RussKeldorph]

Formatting jobs rely on the build to generate a set of compiler command lines (compile_commands.json). Looks like something is going wrong when trying to run cmake to generate that.

[weshaggard]

@RussKeldorph @adiaaida any pointers on where that happens? I cannot find the the script yet. It looks like it is only failing for Release build type for formatting but I cannot see anything yet that would explain why.

[RussKeldorph]

Try running ./build.sh x64 Release configureonly with your change

>./build.sh x64 Release configureonly
Commencing CoreCLR Repo build
Setting up directories for build
Checking prerequisites...
Restoring the OptimizationData package
Running init-tools.sh
Tools are already initialized
Running: /home/russellk/git/coreclr/master/Tools/dotnetcli/dotnet /home/russellk/git/coreclr/master/Tools/run.exe /home/russellk/git/coreclr/master/config.json build -optdata -BuildArch=x64 -BuildType=Release -BuildOS=Linux
Running: /home/russellk/git/coreclr/master/Tools/msbuild.sh /nologo /verbosity:minimal /clp:Summary  /l:BinClashLogger,Tools/net46/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log /t:RestoreOptData  ./build.proj /p:__BuildType=Release /p:__BuildArch=x64 /p:__BuildOS=Linux  /maxcpucount /p:RestoreDefaultOptimizationDataPackage=false /p:UsePartialNGENOptimization=false /p:PortableBuild=true
MSBUILD : error MSB1021: Cannot create an instance of the logger. Could not load file or assembly 'Microsoft.Build.Utilities.v4.0, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

Switch: BinClashLogger,Tools/net46/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log
Command execution failed with exit code 1.
ERROR: An error occured in /home/russellk/git/coreclr/master/Tools/dotnetcli/dotnet /home/russellk/git/coreclr/master/Tools/run 5. Check 5 logs under /home/russellk/git/coreclr/master.
Failed to restore the optimization data package.

[RussKeldorph]

Also, jit-format is in the dotnet/jitutils repo, but I'm 99% certain that if you make the configureonly switch work on build.sh it will fix this formatting problem.

[weshaggard]

OK I was finally able to repro in my ubuntu docker image and figured out what is wrong. I pushed a fix that should disable this logging setting being passed that isn't needed.

[weshaggard]

OK passed the other failure not going to wait on the arm builds as they aren't reliable. I want to kick off an official build to watch.