diff --git a/docs/root/configuration/http_conn_man/headers.rst b/docs/root/configuration/http_conn_man/headers.rst index 21a671d75..9c72fc6ed 100644 --- a/docs/root/configuration/http_conn_man/headers.rst +++ b/docs/root/configuration/http_conn_man/headers.rst @@ -65,13 +65,12 @@ the :option:`--service-node` option. x-envoy-external-address ------------------------ -It is a common case where a service wants to perform analytics based on the client IP address. Per -the lengthy discussion on :ref:`XFF `, this can get -quite complicated. A proper implementation involves forwarding XFF, and then choosing the first non -RFC1918 address *from the right*. Since this is such a common occurrence, Envoy simplifies this by -setting *x-envoy-external-address* during decoding if and only if the request ingresses externally -(i.e., it's from an external client). *x-envoy-external-address* is not set or overwritten for -internal requests. This header can be safely forwarded between internal services for analytics +It is a common case where a service wants to perform analytics based on the origin client's IP +address. Per the lengthy discussion on :ref:`XFF `, +this can get quite complicated, so Envoy simplifies this by setting *x-envoy-external-address* +to the :ref:`trusted client address ` +if the request is from an external client. *x-envoy-external-address* is not set or overwritten +for internal requests. This header can be safely forwarded between internal services for analytics purposes without having to deal with the complexities of XFF. .. _config_http_conn_man_headers_x-envoy-force-trace: @@ -156,8 +155,10 @@ operates in a transparent mode where it does not modify XFF. .. attention:: In general, *use_remote_address* should be set to true when Envoy is deployed as an edge - node, whereas it may need to be set to false when Envoy is used as an internal service node - in a mesh deployment. + node (aka a front proxy), whereas it may need to be set to false when Envoy is used as + an internal service node in a mesh deployment. + +.. _config_http_conn_man_headers_x-forwarded-for_trusted_client_address: The value of *use_remote_address* controls how Envoy determines the *trusted client address*. Given an HTTP request that has traveled through a series of zero or more proxies to reach @@ -166,24 +167,116 @@ accurate. The source IP address of the immediate downstream node's connection to trusted. XFF *sometimes* can be trusted. Malicious clients can forge XFF, but the last address in XFF can be trusted if it was put there by a trusted proxy. -Envoy's rules for determining the trusted client address are: +Envoy's default rules for determining the trusted client address (*before* appending anything +to XFF) are: * If *use_remote_address* is false and an XFF containing at least one IP address is present in the request, the trusted client address is the *last* (rightmost) IP address in XFF. * Otherwise, the trusted client address is the source IP address of the immediate downstream node's connection to Envoy. +In an environment where there are one or more trusted proxies in front of an edge +Envoy instance, the *xff_num_trusted_hops* configuration option can be used to trust +additional addresses from XFF: + +* If *use_remote_address* is false and *xff_num_trusted_hops* is set to a value *N* that is + greater than zero, the trusted client address is the (N+1)th address from the right end + of XFF. (If the XFF contains fewer than N+1 addresses, Envoy falls back to using the + immediate downstream connection's source address as trusted client address.) +* If *use_remote_address* is true and *xff_num_trusted_hops* is set to a value *N* that is + greater than zero, the trusted client address is the Nth address from the right end + of XFF. (If the XFF contains fewer than N addresses, Envoy falls back to using the + immediate downstream connection's source address as trusted client address.) + Envoy uses the trusted client address contents to determine whether a request originated externally or internally. This influences whether the :ref:`config_http_conn_man_headers_x-envoy-internal` header is set. -Testing IPv6 in a large multi-hop system can be difficult from a change management perspective. For -testing IPv6 compatibility of upstream services which parse XFF header values, -:ref:`represent_ipv4_remote_address_as_ipv4_mapped_ipv6 -` -can be enabled in the v2 API. Envoy will append an IPv4 address in mapped IPv6 format, e.g. -::FFFF:50.0.0.1. This change will also apply to -:ref:`config_http_conn_man_headers_x-envoy-external-address`. +Example 1: Envoy as edge proxy, without a trusted proxy in front of it + Settings: + | use_remote_address = true + | xff_num_trusted_hops = 0 + + Request details: + | Downstream IP address = 192.0.2.5 + | XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1" + + Result: + | Trusted client address = 192.0.2.5 (XFF is ignored) + | X-Envoy-External-Address is set to 192.0.2.5 + | XFF is changed to "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5" + | X-Envoy-Internal is removed (if it was present in the incoming request) + +Example 2: Envoy as internal proxy, with the Envoy edge proxy from Example 1 in front of it + Settings: + | use_remote_address = false + | xff_num_trusted_hops = 0 + + Request details: + | Downstream IP address = 10.11.12.13 (address of the Envoy edge proxy) + | XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5" + + Result: + | Trusted client address = 192.0.2.5 (last address in XFF is trusted) + | X-Envoy-External-Address is not modified + | X-Envoy-Internal is removed (if it was present in the incoming request) + +Example 3: Envoy as edge proxy, with two trusted external proxies in front of it + Settings: + | use_remote_address = true + | xff_num_trusted_hops = 2 + + Request details: + | Downstream IP address = 192.0.2.5 + | XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1" + + Result: + | Trusted client address = 203.0.113.10 (2nd to last address in XFF is trusted) + | X-Envoy-External-Address is set to 203.0.113.10 + | XFF is changed to "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5" + | X-Envoy-Internal is removed (if it was present in the incoming request) + +Example 4: Envoy as internal proxy, with the edge proxy from Example 3 in front of it + Settings: + | use_remote_address = false + | xff_num_trusted_hops = 2 + + Request details: + | Downstream IP address = 10.11.12.13 (address of the Envoy edge proxy) + | XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5" + + Result: + | Trusted client address = 203.0.113.10 + | X-Envoy-External-Address is not modified + | X-Envoy-Internal is removed (if it was present in the incoming request) + +Example 5: Envoy as an internal proxy, receiving a request from an internal client + Settings: + | use_remote_address = false + | xff_num_trusted_hops = 0 + + Request details: + | Downstream IP address = 10.20.30.40 (address of the internal client) + | XFF is not present + + Result: + | Trusted client address = 10.20.30.40 + | X-Envoy-External-Address remains unset + | X-Envoy-Internal is set to "true" + +Example 6: The internal Envoy from Example 5, receiving a request proxied by another Envoy + Settings: + | use_remote_address = false + | xff_num_trusted_hops = 0 + + Request details: + | Downstream IP address = 10.20.30.50 (address of the Envoy instance proxying to this one) + | XFF = "10.20.30.40" + + Result: + | Trusted client address = 10.20.30.40 + | X-Envoy-External-Address remains unset + | X-Envoy-Internal is set to "true" A few very important notes about XFF: @@ -206,6 +299,12 @@ A few very important notes about XFF: Envoy will not consider it internal. This is a known "bug" due to the simplification of how XFF is parsed to determine if a request is internal. In this scenario, do not forward XFF and allow Envoy to generate a new one with a single internal origin IP. +3. Testing IPv6 in a large multi-hop system can be difficult from a change management perspective. + For testing IPv6 compatibility of upstream services which parse XFF header values, + :ref:`represent_ipv4_remote_address_as_ipv4_mapped_ipv6 ` + can be enabled in the v2 API. Envoy will append an IPv4 address in mapped IPv6 format, e.g. + ::FFFF:50.0.0.1. This change will also apply to + :ref:`config_http_conn_man_headers_x-envoy-external-address`. .. _config_http_conn_man_headers_x-forwarded-proto: diff --git a/envoy/config/filter/network/http_connection_manager/v2/http_connection_manager.proto b/envoy/config/filter/network/http_connection_manager/v2/http_connection_manager.proto index 31734853c..2066b2569 100644 --- a/envoy/config/filter/network/http_connection_manager/v2/http_connection_manager.proto +++ b/envoy/config/filter/network/http_connection_manager/v2/http_connection_manager.proto @@ -161,7 +161,6 @@ message HttpConnectionManager { // :ref:`config_http_conn_man_headers_x-envoy-external-address` for more information. google.protobuf.BoolValue use_remote_address = 14; - // [#not-implemented-hide:] // The number of additional ingress proxy hops from the right side of the // :ref:`config_http_conn_man_headers_x-forwarded-for` HTTP header to trust when // determining the origin client's IP address. The default is zero if this option