[vs16.7] NuGet Feed Update

[dotnet-bot]

NuGet Feed Update

This pull request updates the usage of NuGet.org in cases where it is used in conjunction with other feeds.

Is this PR required?

Generally yes. If the target branch is no longer in use and will not need to be built in the future, please close this PR.

This PR is broken build, what do I do?

If packages are missing, please tag 'dotnet/dnceng' or 'mmitche' on this PR and note the missing packages.
If there are other unexpected failures, please contact 'dotnet/dnceng'.

[benvillalobos]

Failing on:

D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.20411.9\tools\Tools.proj : error MSB4242: The SDK resolver "Microsoft.DotNet.MSBuildSdkResolver" failed to run. Method not found: 'Microsoft.Build.Framework.SdkResult Microsoft.Build.Framework.SdkResultFactory.IndicateSuccess(System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.Collections.Generic.IDictionary`2<System.String,Microsoft.Build.Framework.SdkResultItem>, System.Collections.Generic.IEnumerable`1<System.String>)'.
D:\a\1\s\stage1\bin\bootstrap\net472\MSBuild\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ILLink.targets(14,3): error MSB4242: The SDK resolver "Microsoft.DotNet.MSBuildSdkResolver" failed to run. Method not found: 'Microsoft.Build.Framework.SdkResult Microsoft.Build.Framework.SdkResultFactory.IndicateSuccess(System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.Collections.Generic.IDictionary`2<System.String,Microsoft.Build.Framework.SdkResultItem>, System.Collections.Generic.IEnumerable`1<System.String>)'.
D:\a\1\s\stage1\bin\bootstrap\net472\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(293,5): error MSB4062: The "NuGet.Build.Tasks.WarnForInvalidProjectsTask" task could not be loaded from the assembly D:\a\1\s\stage1\bin\bootstrap\net472\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.Build.Tasks.dll. Could not load file or assembly 'NuGet.Commands, Version=5.4.0.3, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.20411.9\tools\Tools.proj]
    0 Warning(s)
    3 Error(s)

[benvillalobos]

/cc @mmitche

Having some trouble nailing this down D:\a\1\s\stage1\bin\bootstrap\net472\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(293,5): error MSB4062: The "NuGet.Build.Tasks.WarnForInvalidProjectsTask" task could not be loaded from the assembly D:\a\1\s\stage1\bin\bootstrap\net472\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.Build.Tasks.dll. Could not load file or assembly 'NuGet.Commands, Version=5.4.0.3, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.20411.9\tools\Tools.proj].

I checked both nuget feeds
"https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json", and
"https://api.nuget.org/v3/index.json"

and they both have the right packages in their feeds. Though NuGet.Commands only shows a 5.4.0 version on both.

[mmitche]

That's an odd error for a feed change. Not something I've seen before.

[Forgind]

The error seems a bit different this time around:

.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.20411.9\tools\Tools.proj(0,0): error MSB4242: The SDK resolver "Microsoft.DotNet.MSBuildSdkResolver" failed to run. Method not found: 'Microsoft.Build.Framework.SdkResult Microsoft.Build.Framework.SdkResultFactory.IndicateSuccess(System.String, System.String, System.Collections.Generic.IDictionary`2<System.String,System.String>, System.Collections.Generic.IDictionary`2<System.String,Microsoft.Build.Framework.SdkResultItem>, System.Collections.Generic.IEnumerable`1<System.String>)'.

The method it refers to came in through #5501, which was merged into 16.8 but not 16.7. The normal/easy answer is to update MSBuild so it has the method, but that's a nonstarter here. Instead, I believe we'll need to build with older build tools. If that's right, I'm guessing it's essentially the same problem on earlier versions as well—we'd need to change the build tools to be from the time the particular version came out. That also explains why a later version (i.e., 16.9) was easy to patch.

The one part that's confusing me about that explanation, however, is that it would fail only on Windows Full. It seems like that should make it fail on all legs. Any thoughts?

[Forgind]

I think these are the lines to change? We'd change them to use a specific set of build tools rather than whatever arcade gives us.

[benvillalobos]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[benvillalobos]

What's the order of operations here?

1. We pull the sdk
2. We try to resolve the sdk tools.proj is using
3. We can't resolve it because the resolver we're using from the SDK (note the sdk owns Microsoft.DotNet.MSBuildSdkResolver) is trying to use a method that doesn't exist?

[Forgind]

I think so. Our global.json points to 3.1.100, which (I think) includes MSBuild 16.4, which should be fine. But we have to get that sdk before we can use it, and that's where we're having trouble? I imagine @dsplaisted would know better.

[rainersigwald]

This is failing because the MSBuild under test (16.7) is too old to support the .NET SDK resolver, which came from the installed VS directory--on my machine that's a prerel 16.10; on the official machines it's probably 16.9 RTM. That SDK would be built against MSBuild 16.9 (or higher) and uses #5501.

The fix would be to pin to a specific version of Microsoft.DotNet.MSBuildSdkResolver, which is produced in the SDK and presumably published somewhere. That seems like a good idea in general--all of the "copy LKG from VS into the bootstrap folder" stuff is a bit of a mess and should be removed if possible.

[Forgind]

Were the lines I suggested the right ones to change? Also, from my earlier comment, why is it failing only on Windows full?

[rainersigwald]

Were the lines I suggested the right ones to change?

No, those are unrelated. I'm talking about

msbuild/eng/BootStrapMSBuild.targets

Line 67 in 34d3813

<SdkResolverFiles Include="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Bin\SdkResolvers\Microsoft.DotNet.MSBuildSdkResolver\**\*.*" />

Also, from my earlier comment, why is it failing only on Windows full?

Because that's the only build that populates the bootstrap folder from a VS LKG. The Core flavors use a .NET SDK zip as the lkg/source of truth, and are pinned to a version that is compatible with 16.7.

[Forgind]

What is a(n) LKG?

Were you thinking of pinning the SDK resolver just for servicing branches or also for main? I imagine main isn't pinned right now because we want to be able to use advanced things if we're on an advanced version of MSBuild, and this means it seamlessly tests with the latest bits.

So you're thinking we'd have to download a separate SDK, or is the right SDK on the machine somewhere? If the latter, is this just restricting the Include statement you highlighted to a specific version?

[rainersigwald]

"LKG" means Last Known Good; it's the compiler/SDK/toolchain that is used to build the current version of the compiler/sdk/toolchain. We don't really have the concept formalized--the change required here would formalize it.

The problem is that in order to build our bootstrap folder, we need a source for the non-MSBuild bits. Right now that source is "the toolset that built the first-stage build" but that's not necessarily compatible with the MSBuild that's building, especially if it's from the distant future compared to a servicing branch.

So you're thinking we'd have to download a separate SDK

Yeah, that's my thought. We may not need a full SDK if we can get all the parts we need out of NuGet packages that the SDK repo pushes. Then we can pin that reference to the same release band as the branch that's building. I would do that even for main, but set up a subscription for the SDKs.

Another possible option here would be to drop the bootstrap step for older branches that are failing this way. The official build doesn't bootstrap so failures here wouldn't necessarily block us from getting a build out. Would hate to lose the test coverage, though.

[rainersigwald]

I am tempted to disable the Mono job for this servicing branch. Someone please talk me out of this.

[Forgind]

Mono worked for this branch in the past, so there's no particular reason we should be able to make it work now. It's possible (though looking at the error, very unlikely) that this was random flakiness and everything is good.

I actually think we should disable it, though. We aren't checking mono for master right now, and while it's reasonably possible we'll need a hotfix for 16.7, it's less likely we'll need one for mono. We'd be putting more effort into maintaining msbuild for a smaller segment of customers instead of focusing on big fish.