Skip to content

Support process level access log rate limiting#40992

Merged
kyessenov merged 70 commits intoenvoyproxy:mainfrom
TAOXUY:addAccessLogRateLimiter
Nov 5, 2025
Merged

Support process level access log rate limiting#40992
kyessenov merged 70 commits intoenvoyproxy:mainfrom
TAOXUY:addAccessLogRateLimiter

Conversation

@TAOXUY
Copy link
Copy Markdown
Contributor

@TAOXUY TAOXUY commented Sep 4, 2025

Commit Message: support process level access log rate limiting
Additional Description: this is used to rate limit access log emission on the process level. A typical usage is to rate limit logging to stdout or file when you have multiple access loggers. Fix #40103
Risk Level: NA as this is a new extension
Testing: unit tests and integration test are added
Docs Changes: updated
Release Notes: updated
Platform Specific Features:NA

@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Sep 4, 2025

@kyessenov

Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.h Outdated
@kyessenov
Copy link
Copy Markdown
Contributor

/assign-from @envoyproxy/first-pass-reviewers

Comment thread source/common/access_log/access_log_impl.cc
@kyessenov
Copy link
Copy Markdown
Contributor

/assign @adisuissa
for API review

Comment thread changelogs/current.yaml Outdated
Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.cc Outdated
Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.cc Outdated
Comment thread test/common/access_log/access_log_impl_test.cc Outdated
Copy link
Copy Markdown
Contributor

@kyessenov kyessenov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the lock issue please. Also make sure to add a test that recreates a logger with the same key, since it should have been caught by a test.

@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Sep 5, 2025

Fix the lock issue please. Also make sure to add a test that recreates a logger with the same key, since it should have been caught by a test.

DONE

Copy link
Copy Markdown
Contributor

@adisuissa adisuissa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Added a couple of API comments.

@kyessenov kyessenov self-assigned this Sep 8, 2025
@repokitteh-read-only
Copy link
Copy Markdown

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @adisuissa
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #40992 was synchronize by TAOXUY.

see: more, trace.

Copy link
Copy Markdown
Contributor

@kyessenov kyessenov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic seems right, so it's style questions. @adisuissa for API review.

Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.h Outdated
Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.h Outdated
Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.cc Outdated
Comment thread source/extensions/filters/common/local_ratelimit/local_ratelimit_impl.cc Outdated
Comment thread source/extensions/access_loggers/filters/local_ratelimit/filter.cc Outdated
Comment thread source/extensions/access_loggers/filters/local_ratelimit/filter.h Outdated
Comment thread source/extensions/access_loggers/filters/local_ratelimit/filter.h Outdated
Comment thread source/extensions/access_loggers/filters/local_ratelimit/filter.cc Outdated
Comment thread source/extensions/access_loggers/filters/local_ratelimit/filter.cc Outdated
Copy link
Copy Markdown
Contributor

@adisuissa adisuissa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
I've left a high-level comment.
Please also add integration tests as part of this work.

Comment thread api/envoy/type/v3/token_bucket.proto Outdated
Comment thread tools/code_format/config.yaml Outdated
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Sep 20, 2025

Thanks! I've left a high-level comment. Please also add integration tests as part of this work.

Thx! Integration test has been added.

Signed-off-by: Xuyang Tao <taoxuy@google.com>
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 21, 2025

/retest

@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 23, 2025

Hi @wbpcode are you still the assigned reviewer, could you please take another look? Thx!

wbpcode
wbpcode previously approved these changes Oct 25, 2025
Copy link
Copy Markdown
Member

@wbpcode wbpcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with only one nit comment. Thanks for this contribution.

Comment thread source/extensions/access_loggers/filters/process_ratelimit/filter.cc Outdated
@wbpcode
Copy link
Copy Markdown
Member

wbpcode commented Oct 25, 2025

Could you resolve the conflict also? Thanks.

…ter.cc

Co-authored-by: code <wbphub@gmail.com>
Signed-off-by: Xuyang Tao <taoxuy@google.com>
Signed-off-by: Xuyang Tao <taoxuy@google.com>
Signed-off-by: Xuyang Tao <taoxuy@google.com>
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 25, 2025

/retest

@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 27, 2025

Could you resolve the conflict also? Thanks.

DONE.

@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 27, 2025

LGTM with only one nit comment. Thanks for this contribution.

DONE. Please help merge the PR thx!

@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 28, 2025

@wbpcode

wbpcode
wbpcode previously approved these changes Oct 28, 2025
Copy link
Copy Markdown
Member

@wbpcode wbpcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for your great contribution and all the patience for this works!!!

@wbpcode
Copy link
Copy Markdown
Member

wbpcode commented Oct 28, 2025

Hello, @kyessenov there is a request change from you, could you take a check to that and could we merge it? Thanks.


bool ProcessRateLimitFilter::evaluate(const Formatter::Context&,
const StreamInfo::StreamInfo&) const {
ENVOY_BUG(rate_limiter_->getLimiter() != nullptr,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some metric that shows that access logs are dropped?

Signed-off-by: Xuyang Tao <taoxuy@google.com>
Signed-off-by: Xuyang Tao <taoxuy@google.com>
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 29, 2025

/retest

1 similar comment
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Oct 29, 2025

/retest

Signed-off-by: Xuyang Tao <taoxuy@google.com>
Copy link
Copy Markdown
Contributor

@kyessenov kyessenov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG, one minor error - a duplicated line.

Comment thread source/extensions/access_loggers/filters/process_ratelimit/provider_singleton.cc Outdated
Comment thread test/extensions/access_loggers/filters/process_ratelimit/access_log_impl_test.cc Outdated
Signed-off-by: Xuyang Tao <taoxuy@google.com>
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Nov 4, 2025

/retest

1 similar comment
@TAOXUY
Copy link
Copy Markdown
Contributor Author

TAOXUY commented Nov 4, 2025

/retest

@kyessenov kyessenov merged commit b6ace88 into envoyproxy:main Nov 5, 2025
25 checks passed
grnmeira pushed a commit to grnmeira/envoy that referenced this pull request Mar 20, 2026
…0992)

Commit Message: support process level access log rate limiting
Additional Description: this is used to rate limit access log emission
on the process level. A typical usage is to rate limit logging to stdout
or file when you have multiple access loggers. Fix
envoyproxy#40103
Risk Level: NA as this is a new extension
Testing: unit tests and integration test are added
Docs Changes: updated
Release Notes: updated
Platform Specific Features:NA

---------

Signed-off-by: Xuyang Tao <taoxuy@google.com>
Co-authored-by: code <wbphub@gmail.com>
Signed-off-by: Gustavo <grnmeira@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support token-bucket-style rate limiting for access log emission

5 participants