Support Dynamic HTTP Forward Proxy

[guydc]

Description:
In some cases, the backend address is not known in advance and must be inferred from request/connection attributes (such as a user-provided host header, jwt claim, TLS SNI, etc.).

The Envoy Dynamic Forward proxy feature can be used to achieve dynamic routing. EG can support Dynamic Forward Proxy in the following manner:

• The EG Backend API can be extended to support a "dynamic" endpoint type, representing a dynamic envoy cluster. This is mostly required for BTLS policy attachment.

• The EG BackendTrafficPolicy can be used to enable dynamic routing:

  • Injecting the appropriate HCM filter
  • Defining the strategy for inferring the upstream address
  • Configuring a DNS cache policy

• Implement Dynamic Resolver

• E2E

• Docs

• Add warnings for Dynamic Resolver in docs

• Support appProtocol for Dynamic Resolver

[optional Relevant Links:]

• https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/http_proxy
• https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/sni_dynamic_forward_proxy_filter
• https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/clusters/dynamic_forward_proxy/v3/cluster.proto#envoy-v3-api-msg-extensions-clusters-dynamic-forward-proxy-v3-clusterconfig

[arkodg]

+1

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[arkodg]

@guydc what would the UX of this be like ? a filter like Redirect and DirectResponse or a Backend with a special field set

[rudrakhp]

Maybe a new BackendEndpoint type for dynamic usecases, something like DynamicEndpoint?

[guydc]

Yea, DynamicEndpoint sound ok to me. We need the ability to attach BTLSP and BTP for various cluster-level options, so I think that a filter won't be enough.

However, we also need to inject and configure an HTTP Filter: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/dynamic_forward_proxy/v3/dynamic_forward_proxy.proto#envoy-v3-api-msg-extensions-filters-http-dynamic-forward-proxy-v3-filterconfig

So, we maybe also want to make those settings part of BTP/DynamicEndpoint... BTP is generally better since it's cluster equivalent, while an endpoint is not...

[arkodg]

+1 to a ForwardProxy / DynamicEndpoint type in Backend
raised envoyproxy/envoy#38452 to see if we can create a unique dynamic proxy http filter per xDS Cluster

@guydc how is BTLSP going to work here ? creating a BTLSP per Hostname is going to defeat the purpose here imo

[guydc]

As discussed offline, Envoy supports auto-SAN validation: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/protocol.proto#config-core-v3-upstreamhttpprotocoloptions

The major problem is that BTLSP specs force us to strictly validate the SAN/Hostame specified in BTLSP. I'm ok with adding an autoSANValidation feature in the DynamicEndpoint or in BackendTrafficPolicy.