NTLM credentials not sent by client when there are multiple WWW-Authenticate headers

[aplex]

We have a strange case after migrating from RC1 to RC2. We use HttpClient with default windows credentials authentication. The code works fine with one server, but does not work with another server. In the second case client just receives 401 error and does not start NTLM handshake.

The only difference that I spotted was that second server sends two WWW-Authenticate headers (one with basic, another with HTML).

When I intercepted the response with Fiddler, and removed "WWW-Authenticate" header with basic authorization, everything worked fine, client started NTLM handshake, and finally authorized.

var handler = new HttpClientHandler();
handler.UseDefaultCredentials = true;            
var client = new HttpClient(handler);
HttpResponseMessage response = await client.GetAsync(url);

Request/response from second server:

GET https://my-server:8081/tfs/DefaultCollection/_apis/projects?api-version=1.0 HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Host: my-server:8081


HTTP/1.1 401 Unauthorized
Content-Type: text/html
Server: Microsoft-IIS/8.5
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 3600
Access-Control-Allow-Methods: OPTIONS,GET,POST,PATCH,PUT,DELETE
Access-Control-Expose-Headers: ActivityId,X-TFS-Session,X-MS-ContinuationToken
Access-Control-Allow-Headers: authorization
X-FRAME-OPTIONS: SAMEORIGIN
Set-Cookie: Tfs-SessionId=XXX; path=/; secure
Set-Cookie: Tfs-SessionActive=2016-06-07 20:44:23Z; path=/; secure
WWW-Authenticate: Basic realm="my-server"
WWW-Authenticate: NTLM
P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"
X-Content-Type-Options: nosniff
Date: Tue, 07 Jun 2016 20:44:22 GMT
Content-Length: 1293

OS: windows 10 x64, app running on coreclr RC2

[davidsh]

cc: @stephentoub

Could be related to #17508.

[davidsh]

You code is not turning on default credentials. Is that a typo?

handler.UseDefaultCredentials = false;  

[aplex]

yep, that was a typo. In reality the code is a little bigger - with some user agent headers, and other stuff.

[aplex]

I see this is scheduled for 1.1.0. Is there any workaround available? Maybe there is some place where I could intercept the request and remove the second WWW-Authenticate header before framework handles 401 response?

[davidsh]

@aplex

I tried to repro your scenario but I couldn't repro. I set up an IIS server that uses both ntlm and basic. I was able to connect using default credentials.

One thing I noticed, though, is that my IIS server sends out the 'WWW-Authenticate' headers using NTLM first and then basic:

WWW-Authenticate: NTLM
WWW-Authenticate: Basic realm="testserver"

It shouldn't matter, though, since NTLM will be preferred regardless of what order the server sends them out.

Can you retry your server repro and see if it is different if you have NTLM first and then Basic?

[aplex]

Ok, so I tried to change the order of WWW-Authenticate headers using Fiddler, so that NTLM was first, and client continued with NTLM handshake. I guess it just picks the first header then?

If you need some way to check it, you can use Fiddler with before-request breakpoints that allows to change the 401 response from server before it gets to the client.

[davidsh]

Thx for confirming this. Also, you mentioned that this worked in RC1 but then broke in RC2. Is that correct?

[davidsh]

One other thing to try....are you going thru a proxy? Try turning off proxies completely by setting .UseProxy=false.

I have reproduced a problem with getting headers in the "Basic", "NTLM" order and using a proxy (such as Fiddler). It appears that NTLM handshakes don't go thru.

[davidsh]

cc: @stephentoub @CIPop

This looks like a native WinHTTP bug. I opened up a bug with the WinHTTP team.
Ref: 7924548

[aplex]

Disregard that thing about RC1 - I used full .Net framework there, so that's probably the reason it worked.

I tried with handler.UseProxy = false and result was the same - 401 error

[gerab]

wow, i spent two days with debugger and wireshark to find the same bug. The workaround for me was to offer server all possible authorization schemes:

_credentialCache = new CredentialCache
            {
                {new Uri(ResourceUrls.BaseUrl), "NTLM", _credential},
                {new Uri(ResourceUrls.BaseUrl), "NEGOTIATE", _credential}
            };

or to remove all providers from iis except ntlm. That is not prefered as may cause some issues later .

[karelz]

@davidsh is there something we could do to compensate / detect prior/post the bad behavior in the OS? If we can bring the 2 days tracking time down, that would be amazing ...
@gerab any idea what would have helped you? (beside having the OS bug fixes)

[davidsh]

@gerab Are you going thru a proxy? The original bug report could only be repro'd if going thru a proxy.