SocketHttpHandler does not work with negotiate/kerberos auth on Linux

[wfurt]

I have setup where Linux machine joined domain (using MIT krb5)

clruser@CLRPERFTST003:$ kinit
Password for clradmin@COREFX.NET:
clruser@CLRPERFTST003:$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: clradmin@COREFX.NET

Valid starting Expires Service principal
2018-03-08 14:16:54 2018-03-09 00:16:54 krbtgt/COREFX.NET@COREFX.NET
renew until 2018-03-09 14:16:48
2018-03-08 14:37:52 2018-03-09 00:16:54 HTTP/clrperftst003.corefx.net@COREFX.NET
renew until 2018-03-09 14:16:48

and I verified that curl can get pages using kerberos authentication:

clruser@CLRPERFTST003:~/Downloads/proxy-test/proxy-test$ curl --proxy-nego --proxy-user ':' --proxy CLRperftst003.corefx.net:3128 -v http://mytest.com/

GET http://mytest.com/ HTTP/1.1
Host: mytest.com
Proxy-Authorization: Negotiate YIIFgAYGKwYBBQUCoIIFdDCCBXCgJzAlBgkqhkiG9xIBAgIGBSsFAQUCBgkqhkiC9xIBAgIGBisGAQUCBaKCBUMEggU/YIIFOwYJKoZIhvcSAQICAQBuggUqMIIFJqADAgEFoQMCAQ6iBwMFACAAAACjggQlYYIEITCCBB2gAwIBBaEMGwpDT1JFRlguTkVUoiswKaADAgEDoSIwIBsESFRUUBsYY2xycGVyZnRzdDAwMy5jb3JlZngubmV0o4ID2TCCA9WgAwIBF6EDAgEFooIDxwSCA8O5w6rgIY6bddCzO6I+f0Rke4L137Pv44srNrToc/p3/HhYN8FpjS1uVwKywvekT86iedtMwYEo5TuLs2+1IOMJ+pUjChcqNBvUfwiTMWctTOMHnQN/7FzDBJ0UgZ8336lNx7+OSJLx4td5O1kEDGogUqQZI2A2h0TyOcFfvfDoyhPkMzcmc+3k/7ulgmt72EsoJaYRdnr/HKj8wggsktw5A8EQ2jh3NfSkeMD0Wi/tXEu/45yr3klFCD6JiyGjAIDcNpMTRI4awOPpXSet1J2ET+r7ASXfRRV1FDs1PyExQ3+//2SEdPdBxh96/d3a1ddu3j6Lzk0ybJXe2DMjq8fZmo5zAnZ6iQIzBhgUahtbp2ZSR7s/cMGM9zh0VCZbIkvuj+qogV+uLX3o14XwA7GMvwU9/wlw7AtxYXaMI/I1G7KA5kx+psDouFvEujcQZpuFxEx3sm2voCKuzl0Vr9tETaquXTs7lVuJRBnTmQYg0Erhx5GAaN+09WCR2mZs2mu7CcoeHtgamGxJby1P9Kbht7QWVFZZevuhQLjatbEUWmFUomze52NjcBZD6qjUYbYIHLWTYaJOlRqeSTGSvEDqBTVoEi8DimalFOwfVcUlKyrh5DfaBbDH1WLgLhNxM1XrdFkVvdtH9si68yYays5OnputfmJohaxzRNQza2H8SeoEl3maVD/Tx5Yw3ogkRKXsOhmqK8nNa8DgL1jwgVsLMLRmKZgyl/ysRQVL/pqqMOm5ZVOUDTOcGLMXYAOM/F20Kfy+R/We3cPXL+Jaw/eIGd0Lzrwa3ShNFdImzE6LbHBHwL22IcWWmmVR8ocz7jTX9rIg4HY25CH5Zuv+89V+oyUWKN0x28cS/zjRjlAlvNf0qvJPHcHVZawu5wBanWTWv6CBsPRKQ8QYa/hdllOV8MOD4moNk9QUFJhQndANeMvHKymr8umsRkQgD9gKiN0nxMixHv2CSXrPeBcLmxCiDJerSOHDYw4gGEDMQabfnCmBiRPLZg+sPmZycd49DR1tw/6/HaURnXmPGmyUQ91Sw30VTA+9bQRAYOrrYt0HhPJ6KedBE4OM8DevtrZVzDlNL5SYxbaIElktbdU0r8PxZygDiFYw/lnrW+qTEDB2v+MTHyi1ZK1Fxd+CyDmLIWOqfVbda8vE82kSSbVUJ8tXjWMTp2bSUUFqTZ3xkKyC9HC9BEOODmU/asbdccq5TuGsvhQFTHbfEdl1MI9hQ7vTt1/kOTo5WnGmW7EuLqyIF2yjnjaMnJ1vuLSTN9FsrCHi4+6kgecwgeSgAwIBF6KB3ASB2UEZTU8g7XMA+3tQprxJN7N1Uy80i7fKDpqbnsMUpFj8iQ2go03qzdh/XZQVZKN5bqfSeOVqc3dSHfkyZ+6s+tkDt0hl0KNxGn7AUulUA9LKeGt/ZCQIomooqIbenuq5ioYxpnUJL40pW8HGx6U+NGwHNF+zq5xXenSePW5xXZjhXcOPoJJSbl4e93dD8WWKu7E3gWpoGfOZvIjtTMIXI4e1SpaImE2auRvqqFEBvzx2+7TrSiJL7TurvOM9H6d+y52wFvcz2g9WL827H1O4K0apqtddyNofK70=
User-Agent: curl/7.47.0
Accept: /
Proxy-Connection: Keep-Alive

< HTTP/1.1 200 OK
< Date: Thu, 08 Mar 2018 22:47:54 GMT
< Server: Apache
< Content-Length: 272
< Content-Type: text/html; charset=UTF-8
< X-Cache: MISS from CLRPERFTST003
< X-Cache-Lookup: MISS from CLRPERFTST003:3128
< Via: 1.1 CLRPERFTST003 (squid/3.5.12)
< Connection: keep-alive

When I try simple HTTP client app (#27870) , I get following error. I also try it without explicit credentials and handler.UseDefaultCredentials = true and that simply return 407. It is not clear to me what exactly we expect but With UseDefaultCredential I would expect we pick up Default principal.

System.ComponentModel.Win32Exception (0x80090020): GSSAPI operation failed with error - An invalid status code was supplied (Principal in credential cache does not match desired name).
at System.Net.Security.NegotiateStreamPal.AcquireCredentialsHandle(String package, Boolean isServer, NetworkCredential credential) in /home/clruser/git/corefx/src/Common/src/System/Net/Security/NegotiateStreamPal.Unix.cs:line 313
at System.Net.NTAuthentication.Initialize(Boolean isServer, String package, NetworkCredential credential, String spn, ContextFlagsPal requestedContextFlags, ChannelBinding channelBinding) in /home/clruser/git/corefx/src/Common/src/System/Net/NTAuthentication.Common.cs:line 127
at System.Net.NTAuthentication..ctor(Boolean isServer, String package, NetworkCredential credential, String spn, ContextFlagsPal requestedContextFlags, ChannelBinding channelBinding) in /home/clruser/git/corefx/src/Common/src/System/Net/NTAuthentication.Common.cs:line 98
at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, CancellationToken cancellationToken) in /home/clruser/git/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.NtAuth.cs:line 57
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) in /home/clruser/git/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 255
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken) in /home/clruser/git/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.cs:line 192
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) in /home/clruser/git/corefx/src/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 469
at proxy_test.Program.fetch2(String uri) in /home/clruser/Downloads/proxy-test/proxy-

[davidsh]

Does Negotiate/NTLM auth work to destination servers using SocketsHttpHandler on Linux? I.e., pretend there is no proxy involved. Does auth work, i.e. respond properly to 401's?

It seems to me there should be no difference between 401 and 407 in terms of how the HTTP stack in SocketsHttpHandler works in terms of responding to those auth requests. So, I would guess that if it doesn't work for 407 (where proxies are involved), then it doesn't work either for 401 (server auth).

cc: @geoffkizer

[geoffkizer]

It seems to me there should be no difference between 401 and 407 in terms of how the HTTP stack in SocketsHttpHandler works in terms of responding to those auth requests. So, I would guess that if it doesn't work for 407 (where proxies are involved), then it doesn't work either for 401 (server auth).

Yes, that's right. The only difference is the Proxy-support: Session-Based-Authentication header; if this isn't present on a proxy challenge, we won't attempt to do ntlm/negotiate.

[wfurt]

Is that required @geoffkizer? Squid does not send it and curl works without it.

[geoffkizer]

I don't know. I implemented that because the RFC says it's required. https://tools.ietf.org/html/rfc4559#section-6

That said, the RFC could be full of crap.

If WinHttpHandler and CurlHandler work without the Proxy-support header, then we should just remove that check and match their behavior.

[wfurt]

I set up IIS can enabled "Windows authentication". Curl handler can negotiate with default or explicit credentials. (no proxy) SockerHandler can do neither one. Since I have the setup I can dig little bit deeper.

[davidsh]

The proxy response header, Proxy-support: Session-Based-Authentication, is really matters when doing negotiate/ntlm thru a proxy to a destination server. It doesn't have anything to do really with auth to the proxy itself.

So, is this bug about auth to a proxy, or auth thru a proxy to a destination server, where that server is doing negotiate/ntlm auth?

[geoffkizer]

The proxy response header, Proxy-support: Session-Based-Authentication, is really matters when doing negotiate/ntlm thru a proxy to a destination server. It doesn't have anything to do really with auth to the proxy itself.

Yeah, you are right. I was confused here. The current code is wrong and I'll fix it. This won't fix the above issue, but it will fix #25368 hopefully.

[geoffkizer]

So, is this bug about auth to a proxy, or auth thru a proxy to a destination server, where that server is doing negotiate/ntlm auth?

I'm not sure either. From the trace it looks like proxy auth, but then per #25368 and my comment above, I don't think we're ever responding to 407s today due to the incorrect check for the Proxy-support header.

@wfurt Can you clarify what code is running here and how it fails?

[geoffkizer]

I set up IIS can enabled "Windows authentication". Curl handler can negotiate with default or explicit credentials. (no proxy) SockerHandler can do neither one. Since I have the setup I can dig little bit deeper.

This is on Windows or Linux? What behavior do you see?

[wfurt]

IIS on Windows server 2016. I could set nginx or Apache as well if useful. When running HttpClient on Linux I would see similar error about principal. It seems like with curl handler we let the library deal wth it e.g. we do not use our negotiate code. I'm working on Windows tests and I should results by tomorrow.

[geoffkizer]

I meant, is the client Windows or Linux.

I'm hoping that when running the client on Windows, this succeeds.

When running the client on Linux, it sounds like we have an issue as described above -- exception in AcquireCredentialsHandle. This needs further investigation. Most likely it doesn't matter if it's connection or proxy auth.

[wfurt]

BTW as far as your previous questions: This is client running on Linux. I did bypass the code in question to get to the posted stack trace. Without, it just did not work without any error message - as we ignore proxy failures and it would return 407 response from the proxy. Since I get similar result with authentication to server it suggests some problems with the kerberos layer.

[geoffkizer]

Yeah, there's something screwed up in how we are using the NTAuthentication class on Linux; we need to chase it down.

Do you know if this happens for ntlm as well, or just negotiate?

[wfurt]

NTLM is also broken - tracked in #25368. In curl handler it does not seems to need kerberos at all but we are failing on GSSAPI similar way as negotiate.