Configuring the HttpClientHandler (& Client Certs)

[Tratcher]

What should we add or change to make your life better?

The application needs to be able to configure attributes on the outbound HttpClient / SocketHttpHandler / HttpClientHandler per backend and/or backend endpoint, especially client certificates.

Client certificates for backend endpoints should be configurable in the config file as well as in code.

Why is this important to you?

Service-to-service mutual authentication.

[Tratcher]

#113

[Tratcher]

Note we've now had two partners need to customize server cert validation because of self-signed certs. The workaround at the moment is to replace the entire ProxyHttpClientFactory in DI.

[Tratcher]

Consolidated from #253

Requirements:

• Ability to specify SocketsHttpHandler.SslClientAuthenticationOptions.RemoteCertificateValidationCallback per backend
• Ability for that to be set up to depend on Backend config. Internally, we have an implementation that works efficiently using Signals
  • When a backend's config changes, a derived signal produces a new cert validation callback lambda
  • In the hot-path, cert validation just calls the latest validation lambda produced by the signal, which always tracks the backend config

Why is this important to you?

Compliance requirement. YARP isn't an option for us without this.

[Tratcher]

Also: The ability to add a DelegatingHandler for advanced customization. #283

[samsp-msft]

Need to be able to tweak the handler on a per cluster basis.

Need to consider the association between httpClient and clusters/destinations. Are httpclient instances created on a per cluster basis. What happens if destinations are used in multiple clusters.

[Tratcher]

Island gateway explicitly does not want a destination in multiple clusters to share connections or HttpClient instances. They want to limit the impact different clients/routes can have on each other.

This is the current behavior.

[Tratcher]

Implementation notes:
IProxyHttpClientFactory is split into two APIs, CreateNormalClient() and CreateUpgradableClient(). The only difference is this additional line in the upgrade client:
https://github.com/microsoft/reverse-proxy/blob/24c0c5a969bb025dd7a355f0b62ee4cbac96c99f/src/ReverseProxy/Service/Proxy/Infrastructure/ProxyHttpClientFactory.cs#L63
@stephentoub confirmed that this is only an optimization used by the ClientWebSocket implementation because it knows none of its connections will ever be pooled. We don't need it in YARP.

Proposal: Consolidate to a single CreateClient() API.

Also, we should make these IProxyHttpClientFactory APIs public soon to unblock people, even if they're not the final APIs we want.