KAFKA-7454: Use lazy allocation for SslTransportLayer buffers#5713
Merged
ijuma merged 2 commits intoapache:trunkfrom Sep 29, 2018
Merged
KAFKA-7454: Use lazy allocation for SslTransportLayer buffers#5713ijuma merged 2 commits intoapache:trunkfrom
ijuma merged 2 commits intoapache:trunkfrom
Conversation
junrao
approved these changes
Sep 28, 2018
Contributor
junrao
left a comment
junrao
left a comment
There was a problem hiding this comment.
@rajinisivaram : Thanks for the patch. LGTM
Contributor
Author
|
retest this please |
Member
|
Unrelated test failed in one of the builds (SSL is not enabled in KafkaAdminClientTest.testCreateTopicsRetryBackoff). Merging to trunk and 2.0. There some conflicts when I cherry-picked to 1.1, so I didn't do that. |
ijuma
pushed a commit
that referenced
this pull request
Sep 29, 2018
…l them on close (#5713) Lazy allocation helps when there are a large number of connections that have been accepted, but where no data has been received from the clients. Each buffer is often around 16k (max TLS record size). Nulling the buffers should not make a difference in the current implementation since we release the reference to the channel and transport layer after we close them, but it's a good practice to release medium/large buffers after `close` is called. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
rajinisivaram
added a commit
that referenced
this pull request
Sep 29, 2018
…l them on close (#5713) Lazy allocation helps when there are a large number of connections that have been accepted, but where no data has been received from the clients. Each buffer is often around 16k (max TLS record size). Nulling the buffers should not make a difference in the current implementation since we release the reference to the channel and transport layer after we close them, but it's a good practice to release medium/large buffers after `close` is called. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Contributor
Author
|
Merged to 1.1 as well. |
dhruvilshah3
pushed a commit
to confluentinc/kafka
that referenced
this pull request
Dec 5, 2018
…l them on close (apache#5713) Lazy allocation helps when there are a large number of connections that have been accepted, but where no data has been received from the clients. Each buffer is often around 16k (max TLS record size). Nulling the buffers should not make a difference in the current implementation since we release the reference to the channel and transport layer after we close them, but it's a good practice to release medium/large buffers after `close` is called. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
dhruvilshah3
pushed a commit
to confluentinc/kafka
that referenced
this pull request
Dec 6, 2018
KAFKA-7453: Expire registered channels not selected within idle timeout (apache#5712) KAFKA-7454: Use lazy allocation for SslTransportLayer buffers and null them on close (apache#5713) KAFKA-7304: Limit connection accept rate to prioritize other channels KAFKA-3702: Change log level of SSL close_notify failure
3 tasks
colinhicks
pushed a commit
to confluentinc/kafka
that referenced
this pull request
May 24, 2019
…l them on close (apache#5713) Lazy allocation helps when there are a large number of connections that have been accepted, but where no data has been received from the clients. Each buffer is often around 16k (max TLS record size). Nulling the buffers should not make a difference in the current implementation since we release the reference to the channel and transport layer after we close them, but it's a good practice to release medium/large buffers after `close` is called. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
colinhicks
pushed a commit
to confluentinc/kafka
that referenced
this pull request
May 24, 2019
* KAFKA-3702: Change log level of SSL close_notify failure (apache#5397) SslTransportLayer currently closes the SSL engine and logs a warning if close_notify message canot be sent because the remote end closed its connection. This tends to fill up broker logs, especially when using clients which close connections immediately. Since this log entry is not very useful anyway, it would be better to log at debug level. Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> * KAFKA-7454: Use lazy allocation for SslTransportLayer buffers and null them on close (apache#5713) Lazy allocation helps when there are a large number of connections that have been accepted, but where no data has been received from the clients. Each buffer is often around 16k (max TLS record size). Nulling the buffers should not make a difference in the current implementation since we release the reference to the channel and transport layer after we close them, but it's a good practice to release medium/large buffers after `close` is called. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> * KAFKA-7453: Expire registered channels not selected within idle timeout (apache#5712) Reviewers: Jun Rao <junrao@gmail.com>. Ismael Juma <ismael@juma.me.uk> * KAFKA-7304: Limit connection accept rate to prioritize other channels * Log close of idle connections at `info` * Checkstyle * Fix processor retry logic to advance to the next processor on retry * Add test for connection rate limit * Wrap socketChannel close in try block
pengxiaolong
pushed a commit
to pengxiaolong/kafka
that referenced
this pull request
Jun 14, 2019
…l them on close (apache#5713) Lazy allocation helps when there are a large number of connections that have been accepted, but where no data has been received from the clients. Each buffer is often around 16k (max TLS record size). Nulling the buffers should not make a difference in the current implementation since we release the reference to the channel and transport layer after we close them, but it's a good practice to release medium/large buffers after `close` is called. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Committer Checklist (excluded from commit message)