Handle lists with and without spaces in XFF by rgs1 · Pull Request #3610 · envoyproxy/envoy

rgs1 · 2018-06-12T20:27:50Z

Currently, Utility::getLastAddressFromXFF() only handles lists that
use a comma plus a space as the separator.

Per https://tools.ietf.org/html/rfc7239#section-7.1, spaces are allowed
around commas — so we should handle that too.

This fixes #3607.

Signed-off-by: Raul Gutierrez Segales rgs@pinterest.com

Currently, `Utility::getLastAddressFromXFF()` only handles lists that use a comma plus a space as the separator. Per https://tools.ietf.org/html/rfc7239#section-7.1, spaces are allowed around commas — so we should handle that too. This fixes envoyproxy#3607. Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

mattklein123 · 2018-06-12T20:32:00Z

@rgs1 we are going to merge the revert to try to get master into a good state after some issues. Can you potentially re-apply #3609 in this PR and then @alyssawilk can help review?

rgs1 · 2018-06-12T20:35:07Z

@mattklein123 sure, but do we really need to revert #3609? it's generating valid XFF headers...

mattklein123 · 2018-06-12T20:37:11Z

It is, but when Envoys are paired together, it will no longer work correctly, and I'm sure this will break people in a mesh configuration (including Lyft). I think it's better to revert and reapply as a set.

rgs1 · 2018-06-12T20:37:42Z

@mattklein123 ah true

rgs1 · 2018-06-12T20:38:35Z

i'll add #3609 in a follow-up commit

alyssawilk · 2018-06-12T20:53:46Z

Having the utility fix in the same PR as the change to headers doesn't actually help if you don't roll everything out at once.

I'd honestly be inclined to roll back #3609, check this in, and wait [a month | a full release] before reapplying given there's 0 urgency on removing the whitespace. Alternately we could call it out much more clearly in the release notes that you must replace your upstream Envoys (which allow whitespace and no-whitespace) before replacing downstream Envoys (which change how the header looks) but that seems more dicey.

alyssawilk

Either way this LGTM (modulo nit) - thanks for both debugging and sorting out a fix so quickly!

alyssawilk · 2018-06-12T20:55:38Z

    xff_string = xff_string.substr(last_comma + seperator.size());
  }

+  // ignore the whitespace, since they are allowed in HTTP lists (see rfc7239)


nit: ignore -> Ignore

Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

mattklein123 · 2018-06-12T21:13:55Z

I'd honestly be inclined to roll back #3609, check this in, and wait [a month | a full release] before reapplying given there's 0 urgency on removing the whitespace. Alternately we could call it out much more clearly in the release notes that you must replace your upstream Envoys (which allow whitespace and no-whitespace) before replacing downstream Envoys (which change how the header looks) but that seems more dicey.

This is a good point. Sorry, didn't think about this very much when I mentioned to roll it together. Agre let's apply this first then wait a month or so for the other one. Can we open a tracking issue for that?

mattklein123

Nice, thank you!

mattklein123 · 2018-06-12T22:31:57Z


+  // Ignore the whitespace, since they are allowed in HTTP lists (see RFC7239#section-7.1).
+  xff_string = StringUtil::ltrim(xff_string);
+  xff_string = StringUtil::rtrim(xff_string);


Sorry, just noticed. Do we have a test that covers the need for rtrim?

@mattklein123 no... only ltrim(). Lemme modify the case I added so that it has space at both sides...

@mattklein123 is it fine if I use this test string which should exercise ltrim(), rtrim() and no trim():

"192.0.2.10, 192.0.2.1 ,10.0.0.1"

Or you rather have it broken up in subtests, and be explicit about what path i am exercising?

i guess the above string with comments near each EXPECT call should be clear enough... Let me push what I have and lmk what you think.

@mattklein123 hold it, my comments are in the wrong place, fixing...

@mattklein123 should be good now, sorry about that!

another question

Also, comment about which space trimming case is being tested. Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

mattklein123

great, thanks!

Comments were referring to the wrong test case. Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

alyssawilk · 2018-06-13T14:12:38Z

+
+    // No space trimming.
+    ret = Utility::getLastAddressFromXFF(request_headers, 2);
+    EXPECT_EQ(first_address, ret.address_->ip()->addressAsString());


I think this test would have passed with the previous PR, because we assumed "X, Y" The problem is X,Y with no whitespace on either side.

Can we add one more address in this test with no whitespace before the comma on either side?

Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

rgs1 · 2018-06-13T15:05:13Z

@alyssawilk added — thanks!

rgs1 · 2018-06-13T15:20:01Z

@alyssawilk hmm, do we also need an integration test to check that xff is handled correctly from envoy to envoy (e.g.: something that would have caught the problem with 035a816)?

alyssawilk · 2018-06-13T15:55:33Z

I'd love that, but I don't think we have the ability to set up multi-Envoy integration tests. It's definitely on my list though! I think the best we can do today is an integration test for A,B,C (no whitespace) if you're up for it.

mattklein123 · 2018-06-13T16:01:00Z

Ah thanks @alyssawilk nice catch on the test.

rgs1 mentioned this pull request Jun 12, 2018

Should getLastAddressFromXFF() handle comma-separated lists without spaces too? #3607

Closed

alyssawilk reviewed Jun 12, 2018

View reviewed changes

Fix comment.

cf7bf53

Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

rgs1 mentioned this pull request Jun 12, 2018

Document how to roll out and handle breaking changes to the XFF header #3611

Closed

mattklein123 previously approved these changes Jun 12, 2018

View reviewed changes

mattklein123 reviewed Jun 12, 2018

View reviewed changes

Test rtrim() too.

721400e

Also, comment about which space trimming case is being tested. Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

mattklein123 previously approved these changes Jun 12, 2018

View reviewed changes

mattklein123 self-assigned this Jun 12, 2018

Fix comments.

c999f6f

Comments were referring to the wrong test case. Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

rgs1 dismissed mattklein123’s stale review via c999f6f June 12, 2018 23:18

mattklein123 previously approved these changes Jun 12, 2018

View reviewed changes

alyssawilk reviewed Jun 13, 2018

View reviewed changes

Test address with no space on the left or right.

472bd58

Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>

rgs1 dismissed mattklein123’s stale review via 472bd58 June 13, 2018 15:04

alyssawilk approved these changes Jun 13, 2018

View reviewed changes

mattklein123 approved these changes Jun 13, 2018

View reviewed changes

mattklein123 merged commit 9d3a4eb into envoyproxy:master Jun 13, 2018

rgs1 mentioned this pull request Jun 13, 2018

Add integration test for XFF parsing #3623

Merged

alyssawilk mentioned this pull request Jul 10, 2018

http: removing whitespace from appended XFF headers (take 2) #3828

Closed

Conversation

rgs1 commented Jun 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattklein123 commented Jun 12, 2018

Uh oh!

rgs1 commented Jun 12, 2018

Uh oh!

mattklein123 commented Jun 12, 2018

Uh oh!

rgs1 commented Jun 12, 2018

Uh oh!

rgs1 commented Jun 12, 2018

Uh oh!

alyssawilk commented Jun 12, 2018

Uh oh!

alyssawilk left a comment

Choose a reason for hiding this comment

Uh oh!

alyssawilk Jun 12, 2018

Choose a reason for hiding this comment

Uh oh!

mattklein123 commented Jun 12, 2018

Uh oh!

mattklein123 left a comment

Choose a reason for hiding this comment

Uh oh!

mattklein123 Jun 12, 2018

Choose a reason for hiding this comment

Uh oh!

rgs1 Jun 12, 2018

Choose a reason for hiding this comment

Uh oh!

rgs1 Jun 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rgs1 Jun 12, 2018

Choose a reason for hiding this comment

Uh oh!

rgs1 Jun 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rgs1 Jun 12, 2018

Choose a reason for hiding this comment

Uh oh!

mattklein123 left a comment

Choose a reason for hiding this comment

Uh oh!

alyssawilk Jun 13, 2018

Choose a reason for hiding this comment

Uh oh!

rgs1 commented Jun 13, 2018

Uh oh!

rgs1 commented Jun 13, 2018

Uh oh!

alyssawilk commented Jun 13, 2018

Uh oh!

mattklein123 commented Jun 13, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rgs1 commented Jun 12, 2018 •

edited

Loading

rgs1 Jun 12, 2018 •

edited

Loading

rgs1 Jun 12, 2018 •

edited

Loading