Fix authenticated proxy for dotnet restore

[ptarjan]

Fixes: #123363

Many proxies require the Proxy-Authorization header on the first request rather than sending a 407 Proxy Authentication Required response. This is especially common for:

1. Proxies that drop/ignore unauthenticated connections
2. HTTPS CONNECT tunnels where connection reuse isn't possible
3. Environment variable proxies (HTTP_PROXY/HTTPS_PROXY) with embedded credentials (http://user:password@proxy:port)

This change modifies SendWithProxyAuthAsync to proactively send Basic authentication when credentials are available from the ICredentials provider, rather than waiting for a 407 challenge that may never come.

The reactive authentication flow is still preserved for cases where the proxy does send a 407 response with different auth scheme requirements (Digest, NTLM, Negotiate).

[dotnet-policy-service]

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[ptarjan]

@dotnet-policy-service agree

[ManickaP]

@ptarjan could you please start with filing an issue to kick off a discussion?

There are security implications of pro-actively sending credentials, especially with basic auth (equivalent of plain text) on non-secured connections. So this is not so straightforward and we cannot just accept a PR without any considerations.

[ManickaP]

cc @wfurt

[MihaZupan]

This seems like something that could be driven by the existing SocketsHttpHandler.PreAuthenticate (which is opt-in).

[wfurt]

This change modifies SendWithProxyAuthAsync to proactively send Basic authentication when credentials are available from the ICredentials provider, rather than waiting for a 407 challenge that may never come.

This is problematic IMHO and we will have difficulty with security group IMHO as it especially with basic AUTH leaks the credentials. The "Many proxies" claim seems doubtful as it was generally ok for everyone and the 407 is described in RFC.

I also do not understand the case 2 & 3. I'm not sure how for example environment configuration is related to lack of 407.

having said that I would probably be OK with that if that has some opt-in method. Cutting the rounds can be viewed as general optimization. (as for example noted in #33964)

[ptarjan]

@ptarjan could you please start with filing an issue to kick off a discussion?

#123363

There are security implications of pro-actively sending credentials, especially with basic auth (equivalent of plain text) on non-secured connections. So this is not so straightforward and we cannot just accept a PR without any considerations.

This change modifies SendWithProxyAuthAsync to proactively send Basic authentication when credentials are available from the ICredentials provider, rather than waiting for a 407 challenge that may never come.

This is problematic IMHO and we will have difficulty with security group IMHO as it especially with basic AUTH leaks the credentials. The "Many proxies" claim seems doubtful as it was generally ok for everyone and the 407 is described in RFC.

I also do not understand the case 2 & 3. I'm not sure how for example environment configuration is related to lack of 407.

having said that I would probably be OK with that if that has some opt-in method. Cutting the rounds can be viewed as general optimization. (as for example noted in #33964)

Thanks for the detailed feedback. Let me address each point:

On "many proxies" and RFC compliance:

You're right that RFC 7235 specifies the 407 challenge-response flow. However, the real-world issue is that some proxies (particularly in containerized/cloud environments) don't follow the RFC:

• Case 2 (drop/timeout): Some security-focused proxies silently drop unauthenticated connections rather than responding, to avoid revealing proxy existence to unauthorized clients.
• Case 3 (connection close after 407): For CONNECT tunnels, some proxies close the TCP connection after sending 407, making retry impossible on the same connection. The client would need to open a new connection, but HttpClient's connection pooling makes this complex.

The specific environment where this was reproduced (http://claude.ai/code) uses an egress proxy that returns 401 Unauthorized (not 407) for unauthenticated CONNECT requests. This is non-compliant but real.

On the opt-in approach:

I don't really understand the credential leak, but I'm eager to learn. If the proxy would really like to see your credentials it could return an HTTP 407 and then you will happily send them along, right? Or is this less about an attacker and instead trying to limit logging? If so, then why is the user even using credentials in their proxy url.

I agree an opt-in is more conservative but it does provide user friction.

[wfurt]

any concern here @dotnet/ncl ? Since this would be opt-in I feel it is ..... acceptable. And it would fix #100515 as well. For the 2 cases we have seen so far the AppContext seems sufficient e.g. adding new explicit API would be overkill and it would still depend on 3rd party tools to use it.

[Mondjoe]

Restore

[karelz]

Given that there are design decisions concerns, I would suggest to close this PR or switch it to Draft, until we have consensus.
@ptarjan can you please do one of those actions? Thanks!

[ManickaP]

No need to change into draft, we will take this change. Me and @wfurt will review this.

[ptarjan]

Regarding the scope question: the current implementation applies proactive proxy auth to all proxy requests (both plain proxy and CONNECT tunnels). Should this be restricted to only CONNECT tunnels, or is the general approach acceptable? Happy to narrow the scope if needed.

[ManickaP]

This looks pretty good, @wfurt PTAL as well.

[wfurt]

LGTM. Thanks for the contribution @ptarjan. I think this is very reasonable change ... until we do full API ... if that ever happens.