Skip to content

[MINOR] In-memory stores cleanup#6595

Merged
guozhangwang merged 6 commits intoapache:trunkfrom
ableegoldman:MinorStoreCleanup
Apr 26, 2019
Merged

[MINOR] In-memory stores cleanup#6595
guozhangwang merged 6 commits intoapache:trunkfrom
ableegoldman:MinorStoreCleanup

Conversation

@ableegoldman
Copy link
Copy Markdown
Member

While going through the review of InMemorySessionStore I realized there is also some minor cleanup to be done for the other in-memory stores. This includes trivial changes such as removing unnecessary references to 'this' and moving collection initialization to the declaration. It also fixes some unsafe behavior (registering an iterator from inside its own constructor). In-memory window store iterator classes were made static and some instances of KeyValueIterator missing types were fixed across a handful of tests

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@ableegoldman
Copy link
Copy Markdown
Member Author

Call for review @cadonna @bbejeck @mjsax @vvcephei @guozhangwang

Copy link
Copy Markdown
Member

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ableegoldman,

Thank you for the clean-up!

Made a first pass.


public void close() {
openIterators.remove(this);
callback.deregisterIterator(this);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the object is created with the no-argument constructor, this will throw an NPE.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the benefits of using a callback here than calling openIterators directly? I think adding to openIterators outside of the constructor makes sense, but cannot think of the rationale of doing this upon closure.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I felt it was appropriate to make the iterator class(es) static, in which case we need a way to remove from openIterators on closure. Will fix NPE when I add emptyIterator() method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, thanks for the explanation.


public void close() {
openIterators.remove(this);
callback.deregisterIterator(this);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the benefits of using a callback here than calling openIterators directly? I think adding to openIterators outside of the constructor makes sense, but cannot think of the rationale of doing this upon closure.

@guozhangwang
Copy link
Copy Markdown
Contributor

14:30:43 Execution failed for task ':streams:checkstyleMain'.
14:30:43 > Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/kafka-pr-jdk11-scala2.12/streams/build/reports/checkstyle/main.html
14:30:43   Checkstyle files with violations: 1

Other that that, this PR LGTM.

@ableegoldman
Copy link
Copy Markdown
Member Author

Ack, fixed checkstyle. Waiting for tests to pass

Copy link
Copy Markdown
Member

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thank you @ableegoldman!

@bbejeck
Copy link
Copy Markdown
Member

bbejeck commented Apr 19, 2019

Java 8 passed, Java 11 failed with kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

retest this please

@ableegoldman
Copy link
Copy Markdown
Member Author

Jaba 8 passed, Java 11 failed with test results no longer available

retest this, please

@ableegoldman
Copy link
Copy Markdown
Member Author

Java 11 passed, Java 8 failed on kafka.api.PlaintextConsumerTest.testAsyncCommit

retest this, please

@guozhangwang guozhangwang merged commit dc31fea into apache:trunk Apr 26, 2019
@guozhangwang
Copy link
Copy Markdown
Contributor

Thanks @ableegoldman !

dhruvilshah3 added a commit to confluentinc/kafka that referenced this pull request Apr 29, 2019
* ak/trunk: (42 commits)
  KAFKA-8134: `linger.ms` must be a long
  KAFKA-7779; Avoid unnecessary loop iteration in leastLoadedNode (apache#6081)
  MINOR: Update Gradle to 5.4.1 and update its plugins  (apache#6436)
  MINOR: improve Session expiration notice (apache#6618)
  KAFKA-8029: In memory session store (apache#6525)
  MINOR: In-memory stores cleanup (apache#6595)
  KAFKA-7862 & KIP-345 part-one: Add static membership logic to JoinGroup protocol (apache#6177)
  KAFKA-8254: Pass Changelog as Topic in Suppress Serdes (apache#6602)
  KAFKA-7903: automatically generate OffsetCommitRequest (apache#6583)
  KAFKA-8291 : System test fix (apache#6637)
  MINOR: Do not log retriable offset commit exceptions as errors (apache#5904)
  MINOR: Fix log message error of loadTransactionMetadata (apache#6571)
  MINOR: Fix 404 security features links (apache#6634)
  MINOR: Remove an unnecessary character from broker's startup log
  MINOR: Make LogCleaner.shouldRetainRecord more readable (apache#6590)
  MINOR: Remove implicit return statement (apache#6629)
  KAFKA-8237; Untangle TopicDeleteManager and add test cases (apache#6588)
  KAFKA-8227 DOCS Fixed missing links duality of streams tables (apache#6625)
  MINOR: reformat settings.gradle to be more readable (apache#6621)
  MINOR: Correct RestServerTest formatting
  ...

 Conflicts:
	build.gradle
	settings.gradle
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
While going through the review of InMemorySessionStore I realized there is also some minor cleanup to be done for the other in-memory stores. This includes trivial changes such as removing unnecessary references to 'this' and moving collection initialization to the declaration. It also fixes some unsafe behavior (registering an iterator from inside its own constructor). In-memory window store iterator classes were made static and some instances of KeyValueIterator missing types were fixed across a handful of tests.

Reviewers: Guozhang Wang <wangguoz@gmail.com>,  Bruno Cadonna <bruno@confluent.io>
@ableegoldman ableegoldman deleted the MinorStoreCleanup branch June 26, 2020 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants