Negotiate to NTLM authentication fallback not working

[FixRM]

Description

Hello! I faced the same issue as #28531 but on Windows and net9.0.

Thank you.

Reproduction Steps

I.m using YARP to proxy requests to IIS powered WebApi service that is using Windows authentication. SocketsHttpHandler is used in the following way:

.ConfigureHttpClient((ForwarderHttpClientContext context, SocketsHttpHandler handler) =>
{
    // https://github.com/dotnet/yarp/issues/166#issuecomment-1286334749
    handler.Credentials = CredentialCache.DefaultNetworkCredentials;
})

It works well for instances that have valid Kerberos configuration but it returns 401 if it's not instead of switching to NTLMv2.

Expected behavior

SocketsHttpHandler is autenticated

Actual behavior

I got 401 HTTP response

Regression?

Framework-based HTTP clients seems to be working

Known Workarounds

I can workaround it with custom ICredentials implementation that is forcing to use NTML like this:

public class FallbackToNtmlCredentialManager : ICredentials
{
    public NetworkCredential? GetCredential(Uri uri, string authType)
    {
        if (!string.Equals(authType, "NTLM", StringComparison.InvariantCultureIgnoreCase))
            return null;
        
        return CredentialCache.DefaultNetworkCredentials;        
    }
}

Configuration

net9.0
Windows Server 2016
x64

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[wfurt]

Negotiate should also do NTLM if supported by both sides. You can also try to use https://learn.microsoft.com/en-us/dotnet/api/system.net.credentialcache?view=net-9.0 to avoid custom code.
Depending how it fails I'm not necessarily convinced that SocketsHttpHandler should keep falling back to less secure mechanisms. (even if Framework does)

[dotnet-policy-service]

This issue has been marked needs-author-action and may be missing some important information.

[dotnet-policy-service]

This issue has been automatically marked no-recent-activity because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will remove no-recent-activity.

[dotnet-policy-service]

This issue will now be closed since it had been marked no-recent-activity but received no further activity in the past 14 days. It is still possible to reopen or comment on the issue, but please note that the issue will be locked if it remains inactive for another 30 days.