Skip to content

KAFKA-8736: Streams performance improvement, use isEmpty() rather tha…#7164

Merged
mjsax merged 1 commit intoapache:trunkfrom
mjarvie:streams_performance_size
Aug 6, 2019
Merged

KAFKA-8736: Streams performance improvement, use isEmpty() rather tha…#7164
mjsax merged 1 commit intoapache:trunkfrom
mjarvie:streams_performance_size

Conversation

@mjarvie
Copy link
Copy Markdown
Contributor

@mjarvie mjarvie commented Aug 5, 2019

…n size() == 0

According to https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentSkipListMap.html#size--, the size method has to traverse all elements to get a count. It looks like the count is being compared against 0 to determine if the map is empty; In this case, we don't need a full count. Instead, the isEmpty() method should be used, which just looks for one node.

No expected changes to unit or integration tests.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@ableegoldman
Copy link
Copy Markdown
Member

Thanks for the PR! LGTM.

I think this should be cherry-picked back to 2.3 cc/ @bbejeck @mjsax @guozhangwang

@mjsax mjsax added the streams label Aug 5, 2019
Copy link
Copy Markdown
Member

@mjsax mjsax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Waiting for Jenkins to finish before we can merge.

Copy link
Copy Markdown
Contributor

@guozhangwang guozhangwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mjarvie ! LGTM.

@mjsax
Copy link
Copy Markdown
Member

mjsax commented Aug 5, 2019

Java 11 / 2.12 failed with kafka.server.DescribeLogDirsRequestTest.testDescribeLogDirsRequest
Java 11 / 2.13 failed with kafka.server.DescribeLogDirsRequestTest.testDescribeLogDirsRequest
Java 8 failed with:

org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSourceConnector
org.apache.kafka.connect.integration.RebalanceSourceConnectorsIntegrationTest.testStartTwoConnectors
kafka.api.SaslSslAdminClientIntegrationTest.testIncrementalAlterConfigsForLog4jLogLevels
kafka.api.SaslSslAdminClientIntegrationTest.testIncrementalAlterConfigsForLog4jLogLevelsCanResetLoggerToCurrentRoot

Retest this please

@mjsax
Copy link
Copy Markdown
Member

mjsax commented Aug 6, 2019

Java 11 / 2.12: org.apache.kafka.connect.integration.RebalanceSourceConnectorsIntegrationTest.testDeleteConnector
Java 11 / 2.13: kafka.server.DescribeLogDirsRequestTest.testDescribeLogDirsRequest
Java 8: kafka.server.DescribeLogDirsRequestTest.testDescribeLogDirsRequest

Merging. Thanks for the PR @mjarvie

@mjsax mjsax merged commit 7ebcd50 into apache:trunk Aug 6, 2019
mjsax pushed a commit that referenced this pull request Aug 6, 2019
…n size() == 0 (#7164)

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
mjsax pushed a commit that referenced this pull request Aug 6, 2019
…n size() == 0 (#7164)

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
mjsax pushed a commit that referenced this pull request Aug 6, 2019
…n size() == 0 (#7164)

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
mjsax pushed a commit that referenced this pull request Aug 6, 2019
…n size() == 0 (#7164)

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
@mjsax
Copy link
Copy Markdown
Member

mjsax commented Aug 6, 2019

Merged to trunk and cherry-picked to 2.3, 2.2, 2.1, and 2.0.

ijuma added a commit to confluentinc/kafka that referenced this pull request Aug 8, 2019
The avro upgrade was needed to fix the following error during
':support-metrics-client:generateAvro':

Caused by: org.apache.velocity.exception.MethodInvocationException: Variable $velocityCount has not been set at /org/apache/avro/compiler/specific/templates/java/classic/record.vm[line 82, column 150]

Conflicts:
 * gradle.properties -> trivial fix, `scalaVersion` is next to `version`, which
 is different in ccs kafka
 * gradle/dependencies.gradle -> reduce divergence with apache kafka by using
 `httpclient` instead of `httpcomponents` from the `versions` array. Remove
 unused `httpmime` and `httpcomponents` from said array.

* apache-github/trunk:
  MINOR: Update dependencies for Kafka 2.4 (apache#7126)
  KAFKA-8599: Use automatic RPC generation in ExpireDelegationToken
  MINOR: Upgrade jackson-databind to 2.9.9.3 (apache#7125)
  MINOR: some small style fixes to RoundRobinPartitioner
  KAFKA-8736: Streams performance improvement, use isEmpty() rather than size() == 0 (apache#7164)
  Minor: Refactor methods to add metrics to sensor in `StreamsMetricsImpl` (apache#7161)
xiowu0 pushed a commit to linkedin/kafka that referenced this pull request Aug 22, 2019
…t, use isEmpty() rather than size() == 0 (apache#7164)

TICKET = KAFKA-8736
LI_DESCRIPTION =
EXIT_CRITERIA = HASH [0826d6e]
ORIGINAL_DESCRIPTION =

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
(cherry picked from commit 0826d6e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants