Official Build Failures in dotnet/installer when publishing

[JohnTortugo]

• This issue is blocking
• This issue is causing unreasonable pain

Failures in the official build of the master branch of dotnet/installer.

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Builds with the issue:

https://dev.azure.com/dnceng/internal/_build/results?buildId=611877&view=results
https://dev.azure.com/dnceng/internal/_build/results?buildId=611784&view=results
https://dev.azure.com/dnceng/internal/_build/results?buildId=611508&view=results
https://dev.azure.com/dnceng/internal/_build/results?buildId=611468&view=results
https://dev.azure.com/dnceng/internal/_build/results?buildId=611427&view=results
https://dev.azure.com/dnceng/internal/_build/results?buildId=610722&view=results

[markwilkie]

cc/ @ilyas1974

[MattGal]

From reading several logs, the failure here is consistently right when this is called (container ExistsAsync()):
https://github.com/emgarten/Sleet/blob/master/src/SleetLib/FileSystem/AzureFileSystem.cs#L71

I am chasing down a repro now. I have a couple theories and will update the issue as I find out.

[JohnTortugo]

Un-assigning myself as I'm investing all my time on this other issue: #5328

[MattGal]

I believe it was updating the .NET runtime version used here: dotnet/installer@2db1aa8 that broke this. It's the first commit this occurred, as well as it updates the dotnet SDK version used.

To evaluate whether this is accurate, I'm queueing a test build which won't publish with that line rolled back to see what happens.

[MattGal]

Rolled back that single line with the first commit hash that had the problem in this build: https://dev.azure.com/dnceng/internal/_build/results?buildId=612622&view=results

[MattGal]

That didn't work; the other changes were dependent on the newer SDK. I am now rebuilding (with only dev publishing enabled) the previous good commit. This will confirm that the commit (updating dependencies from dotnet/sdk ) was the cause if it's green; Once that's known we can discuss what to do, since we don't actually build the API that's blowing up.

[markwilkie]

Head's up @jaredpar and @marcpopMSFT: Looks like we likely have another "chicken and egg" situation brewing w/ the latest SDK that Arcade is pointing to. Meaning....this'll come in as a breaking change to installer when we promote. (tomorrow??)

I think this problem will actually get sweeped under the rug once we promote Arcade, as the payload includes the changes to stop publishing to the dotnet-core sleet feed, so it won't be that noisy.

Still probably worth understanding how version 5.0.100-preview.4.20217.10 of the SDK is breaking the Azure APIs that we're using, if that's indeed the culprit.

[MattGal]

Update this morning; forgot to add a channel for that update, so spun again