Skip to content

stats: Fix gRPC xDS pending_requests: now drains back to 0#5796

Merged
htuch merged 10 commits intoenvoyproxy:masterfrom
fredlas:FIX_pending_requests
Feb 11, 2019
Merged

stats: Fix gRPC xDS pending_requests: now drains back to 0#5796
htuch merged 10 commits intoenvoyproxy:masterfrom
fredlas:FIX_pending_requests

Conversation

@fredlas
Copy link
Copy Markdown
Contributor

@fredlas fredlas commented Jan 31, 2019

Description: xDS gRPC's stat for number of xDS request queued to be sent was previously only getting set when an attempt to send hit rate limiting. In addition to not updating when new items arrive, this stat would never return to 0 once set to non-0.

Risk Level: low
Testing: added coverage of fixed thing to existing unit test

Signed-off-by: Fred Douglas <fredlas@google.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
@fredlas
Copy link
Copy Markdown
Contributor Author

fredlas commented Jan 31, 2019

Two notes: 1) I intend to move checkRateLimitAllowsDrain() to private; not doing it yet to keep the more important changes easy to see in the diff, and 2) I don't understand why the test isn't passing. I think I either don't exactly understand what the test harness is doing, or stats_store_.gauge().value() does not refer to the same stats store I'm causing a change to.

htuch
htuch previously approved these changes Feb 1, 2019
Copy link
Copy Markdown
Member

@htuch htuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ta!

@htuch
Copy link
Copy Markdown
Member

htuch commented Feb 1, 2019

@fredlas can you look into the build failures? Seems legit. Feel free to make the helper private as needed.

Signed-off-by: Fred Douglas <fredlas@google.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
@fredlas
Copy link
Copy Markdown
Contributor Author

fredlas commented Feb 4, 2019

Turns out I was applying the test framework correctly, and just not understanding the logic being tested! I now realize that the request queue only gets used in cases of rate limiting. Left a comment to clarify.

Signed-off-by: Fred Douglas <fredlas@google.com>
@fredlas
Copy link
Copy Markdown
Contributor Author

fredlas commented Feb 4, 2019

After working with the guts of stats in another PR, I realized why this one was originally using set() like it was: it was avoiding activating this stat in the (common) case that rate limiting was not active. I switched from inc/dec back to set. But, the set() is now in a different place, where it will be correct.

Comment thread source/common/config/grpc_stream.h
Signed-off-by: Fred Douglas <fredlas@google.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
@htuch
Copy link
Copy Markdown
Member

htuch commented Feb 4, 2019

Please merge master to pick up #5827.

Signed-off-by: Fred Douglas <fredlas@google.com>
@fredlas
Copy link
Copy Markdown
Contributor Author

fredlas commented Feb 4, 2019

Merged, thanks for the heads up.

@fredlas
Copy link
Copy Markdown
Contributor Author

fredlas commented Feb 6, 2019

Ping; should be ready to go now.

Comment thread source/common/config/grpc_stream.h Outdated
Comment thread source/common/config/grpc_stream.h Outdated
@htuch htuch added the waiting label Feb 8, 2019
Signed-off-by: Fred Douglas <fredlas@google.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
Copy link
Copy Markdown
Member

@htuch htuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mattklein123
Copy link
Copy Markdown
Member

@fredlas I don't see any test change in the diff? Did a change get dropped accidentally?

@htuch
Copy link
Copy Markdown
Member

htuch commented Feb 11, 2019

@mattklein123 I see comment changes in 7998d8d, which covers my last set of Qs.

@htuch htuch merged commit cdcdfa6 into envoyproxy:master Feb 11, 2019
@mattklein123
Copy link
Copy Markdown
Member

@htuch Shouldn't this change have a test that failed before the fix? I don't see any test changes?

@fredlas
Copy link
Copy Markdown
Contributor Author

fredlas commented Feb 11, 2019

Whoops, yeah. I started with tests that were testing what I (wrongly) thought the logic was supposed to be, and removed them once I realized I was testing the wrong thing, but didn't add anything else. #5903 tests the fixed behavior. Sorry!

@fredlas fredlas deleted the FIX_pending_requests branch February 11, 2019 15:30
mattklein123 pushed a commit that referenced this pull request Feb 11, 2019
Signed-off-by: Fred Douglas <fredlas@google.com>
fredlas added a commit to fredlas/envoy that referenced this pull request Mar 5, 2019
…y#5796)

xDS gRPC's stat for number of xDS request queued to be sent was previously only getting set when an attempt to send hit rate limiting. In addition to not updating when new items arrive, this stat would never return to 0 once set to non-0.

Risk Level: low
Testing: added coverage of fixed thing to existing unit test

Signed-off-by: Fred Douglas <fredlas@google.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
fredlas added a commit to fredlas/envoy that referenced this pull request Mar 5, 2019
…xy#5903)

Signed-off-by: Fred Douglas <fredlas@google.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants