-
Notifications
You must be signed in to change notification settings - Fork 5.4k
[docs] original src listener filter docs #5539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
025eaaf
Add documentation for original_src filter
klarose b8b7ce6
Fix bad ordering for docs build
klarose 69f6c5f
Misc docs fixes
klarose 20f2ce2
Add IP Transparency documentation
klarose 572a2c6
Mention route_localnet
klarose f1dceb3
Merge branch 'master' into original_src_docs
klarose 0f0a59a
Add some references to docs
klarose File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,3 +10,4 @@ Filters | |
| thrift/thrift | ||
| accesslog/v2/accesslog.proto | ||
| fault/v2/fault.proto | ||
| listener/listener | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| Listener filters | ||
| ================ | ||
|
|
||
| .. toctree:: | ||
| :glob: | ||
| :maxdepth: 2 | ||
|
|
||
| */v2alpha1/* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
75 changes: 75 additions & 0 deletions
75
docs/root/configuration/listener_filters/original_src_filter.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| .. _config_listener_filters_original_src: | ||
|
|
||
| Original Source | ||
| =============== | ||
|
|
||
| * :ref:`Listener filter v2 API reference <envoy_api_msg_config.filter.listener.original_src.v2alpha1.OriginalSrc>` | ||
| * This filter should be configured with the name *envoy.listener.original_src*. | ||
|
|
||
| The original source listener filter replicates the downstream remote address of the connection on | ||
| the upstream side of Envoy. For example, if a downstream connection connects to Envoy with IP | ||
| address ``10.1.2.3``, then Envoy will connect to the upstream with source IP ``10.1.2.3``. | ||
|
|
||
| Interaction with Proxy Protocol | ||
| -------------------------------- | ||
|
|
||
| If the connection has not had its source address translated or proxied, then Envoy can simply use | ||
| the existing connection information to build the correct downstream remote address. However, if this | ||
| is not true, a :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>` may be used to | ||
| extract the downstream remote address. | ||
|
|
||
| IP Version Support | ||
| ------------------ | ||
| The filter supports both IPv4 and IPv6 as addresses. Note that the upstream connection must support | ||
| the version used. | ||
|
|
||
| Extra Setup | ||
| ----------- | ||
|
|
||
| The downstream remote address used will likely be globally routable. By default, packets returning | ||
| from the upstream host to that address will not route through Envoy. The network must be configured | ||
| to forcefully route any traffic whose IP was replicated by Envoy back through the Envoy host. | ||
|
|
||
| If Envoy and the upstream are on the same host -- e.g. in an sidecar deployment --, then iptables | ||
| and routing rules can be used to ensure correct behaviour. The filter has an unsigned integer | ||
| configuration, | ||
| :ref:`mark <envoy_api_field_config.filter.listener.original_src.v2alpha1.OriginalSrc.mark>`. Setting | ||
| this to *X* causes Envoy to *mark* all upstream packets originating from this listener with value | ||
| *X*. | ||
|
|
||
| We can use the following set of commands to ensure that all ipv4 and ipv6 traffic marked with *X* | ||
| (assumed to be 123 in the example) routes correctly. Note that this example assumes that *eth0* is | ||
| the default outbound interface. | ||
|
|
||
| .. code-block:: text | ||
|
|
||
| iptables -t mangle -I PREROUTING -m mark --mark 123 -j CONNMARK --save-mark | ||
| iptables -t mangle -I OUTPUT -m connmark --mark 123 -j CONNMARK --restore-mark | ||
| ip6tables -t mangle -I PREROUTING -m mark --mark 123 -j CONNMARK --save-mark | ||
| ip6tables -t mangle -I OUTPUT -m connmark --mark 123 -j CONNMARK --restore-mark | ||
| ip rule add fwmark 123 lookup 100 | ||
| ip route add local 0.0.0.0/0 dev lo table 100 | ||
| ip -6 rule add fwmark 123 lookup 100 | ||
| ip -6 route add local ::/0 dev lo table 100 | ||
| echo 1 > /proc/sys/net/ipv4/conf/eth0/route_localnet | ||
|
|
||
|
|
||
| Example Listener configuration | ||
| ------------------------------ | ||
|
|
||
| The following example configures Envoy to use the original source for all connections made on port | ||
| 8888. It uses Proxy Protocol to determine the downstream remote address. All upstream packets are | ||
| marked with 123. | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| listeners: | ||
| - address: | ||
| socket_address: | ||
| address: 0.0.0.0 | ||
| port_value: 8888 | ||
| listener_filters: | ||
| - name: envoy.listener.proxy_protocol | ||
| - name: envoy.listener.original_src | ||
| config: | ||
| mark: 123 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,3 +39,4 @@ Architecture overview | |
| scripting | ||
| ext_authz_filter | ||
| overload_manager | ||
| ip_transparency | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| .. _arch_overview_ip_transparency: | ||
|
|
||
| IP Transparency | ||
| =============== | ||
|
|
||
| What is IP Transparency | ||
| ----------------------- | ||
|
|
||
| As a proxy, Envoy is an IP endpoint: it has its own IP address, distinct from that of any downstream | ||
| requests. Consequently, when Envoy establishes connections to upstream hosts, the IP address of that | ||
| connection will be different from that of any proxied connections. | ||
|
|
||
| Sometimes the upstream server or network may need to know the original IP address of the connection, | ||
| called the *downstream remote address*, for many reasons. Some examples include: | ||
|
|
||
| * the the IP address being used to form part of an identity, | ||
| * the IP address being used to enforce network policy, or | ||
| * the IP address being included in an audit. | ||
|
|
||
| Envoy supports multiple methods for providing the downstream remote address to the upstream host. | ||
| These techniques vary in complexity and applicability. | ||
|
|
||
| HTTP Headers | ||
| ------------ | ||
|
|
||
| HTTP headers may carry the original IP address of the request in the | ||
| :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>` header. The upstream server | ||
| can use this header to determine the downstream remote address. | ||
|
|
||
| The HTTP header approach has a few downsides: | ||
|
|
||
| * It is only applicable to HTTP. | ||
| * It may not be supported by the upstream host. | ||
| * It requires careful configuration. | ||
|
|
||
| Proxy Protocol | ||
| -------------- | ||
|
|
||
| `HAProxy Proxy Protocol <http://www.haproxy.org/download/1.9/doc/proxy-protocol.txt>`_ defines a | ||
| protocol for communicating metadata about a connection over TCP, prior to the main TCP stream. This | ||
| metadata includes the source IP. Envoy supports consuming this information using | ||
| :ref:`Proxy Protocol filter <config_listener_filters_proxy_protocol>`, which may be used to recover | ||
| the downstream remote address for propagation into an | ||
| :ref:`x-forwarded-for <config_http_conn_man_headers_x-forwarded-for>` header. It can also be used in | ||
| conjunction with the | ||
| :ref:`Original Src Listener Filter <arch_overview_ip_transparency_original_src_listener>`. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: can we expand it to full name i.e |
||
|
|
||
| Some drawbacks to Proxy Protocol: | ||
|
|
||
| * It only supports TCP protocols. | ||
| * It requires upstream host support. | ||
| * Envoy cannot yet send it to the upstream. | ||
|
|
||
| .. _arch_overview_ip_transparency_original_src_listener: | ||
|
|
||
| Original Source Listener Filter | ||
| ------------------------------- | ||
|
|
||
| In controlled deployments, it may be possible to replicate the downstream remote address on the | ||
| upstream connection by using a | ||
| :ref:`Original Source listener filter <config_listener_filters_original_src>`. No metadata is added | ||
| to the upstream request or stream. Rather, the upstream connection itself will be established with | ||
| the downstream remote address as its source address. This filter will work with any upstream | ||
| protocol or host. However, it requires fairly complex configuration, and it may not be supported in | ||
| all deployments due to routing constraints. | ||
|
|
||
| Some drawbacks to the Original Source filter: | ||
|
|
||
| * It requires that Envoy have access to the downstream remote address. | ||
| * Its configuration is relatively complex. | ||
| * It may introduce a slight performance hit due to restrictions on connection pooling. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for putting together these docs @klarose, they're very helpful.
Do you happen to know how the PREROUTING rule below works for local packets?
All the iptables diagrams seem to show no path from local process to PREROUTING so I'm having a hard time understanding how this rule applies to local traffic with the sidecar.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It has been a while since I last did this, so I'm a little rusty. As some background, this configuration was inspired by mmproxy. They don't explain why exactly. :)
I think that it has to do with the fact that there is a separate routing table involved. Perhaps the system considers transit between different routing tables as transit between different networks?
It could also be that the kernel, seeing that the source IP isn't a local one, treats it as having come from an external network.
I'm sorry I don't have a better answer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That helps, thanks!