[CurlHandler] curl HttpHandler can only do basic auth to proxy

[wfurt]

I have proxy setup with supporting only NTLM authentication and I have simple app fetching given URL through the proxy

        static async Task fetch2(String uri)
        {
            HttpClientHandler handler = new HttpClientHandler();
            handler.PreAuthenticate = false;
            handler.Proxy = new WebProxy("clrperftst003:3128");
            handler.Proxy.Credentials = new NetworkCredential("foo", "bar", "corefx");
            var client = new HttpClient(handler);
            var result1 = await client.GetAsync(uri);
            Console.WriteLine(result1);
    }

This work when running on Windows but always fails on Ubuntu16.

It seems like the code always sends Basic auth when credentials are available.
When that fails it never tries again with schema supported by the server.
Also note, that the PreAuthenticate has no impact so the client will always leak plaintext password.
I tested this with server supporting Digest or NTLM and behavior is identical.

Failed request/response from c#

GET http://mytest.com/ HTTP/1.1
Host: mytest.com
Proxy-Authorization: Basic Y29yZWZ4XGZvbzpiYXI=
Accept: /
Proxy-Connection: Keep-Alive

HTTP/1.1 407 Proxy Authentication Required
Server: squid/3.5.12
Mime-Version: 1.0
Date: Thu, 08 Mar 2018 21:26:30 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3566
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: en
Proxy-Authenticate: NTLM
X-Cache: MISS from CLRPERFTST003
X-Cache-Lookup: NONE from CLRPERFTST003:3128
Via: 1.1 CLRPERFTST003 (squid/3.5.12)
Connection: keep-alive

On same system, using same setup, curl can work just fine. (the credentials are different but I did not want to leak my test creds via basic auth)

clruser@CLRPERFTST003:~/Downloads/proxy-test/proxy-test$ curl -v --proxy-ntlm http://mytest.com

• Rebuilt URL to: http://mytest.com/
• Trying 10.194.114.94...
• Connected to clrperftst003 (10.194.114.94) port 3128 (#0)
• Proxy auth using NTLM with user 'corefx\clradmin'

GET http://mytest.com/ HTTP/1.1
Host: mytest.com
Proxy-Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA=
User-Agent: curl/7.47.0
Accept: /
Proxy-Connection: Keep-Alive

< HTTP/1.1 407 Proxy Authentication Required
< Server: squid/3.5.12
< Mime-Version: 1.0
< Date: Thu, 08 Mar 2018 21:30:40 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3624
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Vary: Accept-Language
< Content-Language: en
< Proxy-Authenticate: NTLM TlRMTVNTUAACAAAABgAGADgAAAAGgokC34SGakN19rQAAAAAAAAAAIoAigA+AAAABgEAAAAAAA9DT1JFRlgCAAwAQwBPAFIARQBGAFgAAQAaAEMATABSAFAARQBSAEYAVABTAFQAMAAwADMABAAUAGMAbwByAGUAZgB4AC4AbgBlAHQAAwAwAGMAbAByAHAAZQByAGYAdABzAHQAMAAwADMALgBjAG8AcgBlAGYAeAAuAG4AZQB0AAcACAC4FcK0JLfTAQAAAAA=
< X-Cache: MISS from CLRPERFTST003
< X-Cache-Lookup: NONE from CLRPERFTST003:3128
< Via: 1.1 CLRPERFTST003 (squid/3.5.12)
< Connection: keep-alive
<

• Ignoring the response-body
• Connection #0 to host clrperftst003 left intact
• Issue another request to this URL: 'http://mytest.com/'
• Found bundle for host mytest.com: 0x5593227ec670 [can pipeline]
• Re-using existing connection! (#0) with proxy clrperftst003
• Connected to clrperftst003 (10.194.114.94) port 3128 (#0)
• Proxy auth using NTLM with user 'corefx\clradmin'

GET http://mytest.com/ HTTP/1.1
Host: mytest.com
Proxy-Authorization: NTLM TlRMTVNTUAADAAAAGAAYAEAAAAC6ALoAWAAAAAYABgASAQAACAAIABgBAAANAA0AIAEAAAAAAAAAAAAABoKJAjaw4BUrm5vWknDqlbYDMg3LFocJDoCzj8WVcO2Oe84IffaBmcn049UBAQAAAAAAAABAkbQkt9MByxaHCQ6As48AAAAAAgAMAEMATwBSAEUARgBYAAEAGgBDAEwAUgBQAEUAUgBGAFQAUwBUADAAMAAzAAQAFABjAG8AcgBlAGYAeAAuAG4AZQB0AAMAMABjAGwAcgBwAGUAcgBmAHQAcwB0ADAAMAAzAC4AYwBvAHIAZQBmAHgALgBuAGUAdAAHAAgAuBXCtCS30wEAAAAAAAAAAGNvcmVmeGNscmFkbWluQ0xSUEVSRlRTVDAwMw==
User-Agent: curl/7.47.0
Accept: /
Proxy-Connection: Keep-Alive

< HTTP/1.1 200 OK
< Date: Thu, 08 Mar 2018 21:30:40 GMT
< Server: Apache
< Content-Length: 271
< Content-Type: text/html; charset=UTF-8
< X-Cache: MISS from CLRPERFTST003
< X-Cache-Lookup: MISS from CLRPERFTST003:3128
< Via: 1.1 CLRPERFTST003 (squid/3.5.12)
< Connection: keep-alive
<

Error. Page cannot be displayed. Please contact your service provider for more details. (5)

* Connection #0 to host clrperftst003 left intact

[geoffkizer]

@wfurt, are you using the existing NTLM/Negotiate tests in DefaultCredentialsTests.cs? I know we have scripts to set up the environment to run these tests in, though they may need some work. @davidsh would know what's needed here.

Seems like the first step in NT auth testing should be getting these tests running and passing; we can add additional variations from there.

[wfurt]

no. I have simple app command line app (like curl in c#) and I use that to test with real servers. One of my goals was to observe difference between handlers. I'll get the NTLM tests running but that can be only subset of existing functionality @geoffkizer

[geoffkizer]

I'll get the NTLM tests running but that can be only subset of existing functionality

Right, I'm not suggesting that's all we need to do.

I just want to make sure that
(a) we're leveraging whatever existing assets we have here (i.e. the existing scripts for setting up servers for NT auth again regular servers), unless we don't think it's worthwhile for some reason
(b) we have a roadmap for enabling multimachine testing for not just NT auth but for other scenarios where we could or should use it

I know @davidsh has a lot of experience with these testing scenarios. If you guys are already on the same page here, then great. If not, can you sync up?

[karelz]

@wfurt is it important for 2.1? It does not look like that from the description.

[davidsh]

@wfurt is it important for 2.1? It does not look like that from the description.

IMHO, proxy authentication in enterprise scenarios tends to use Negotiate/NTLM much more than basic auth. So, getting proxy auth to work with schemes beyond basic is important.

[wfurt]

I can retest with 2.0 to see if this is regression.

[wfurt]

shall we close this @karelz since 2.1 fixes it?

[karelz]

Agreed, 2.1 supports all authentications via SocketsHttpHandler. Closing