Skip to content

KAFKA-7660: fix streams and Metrics memory leaks#5980

Merged
mjsax merged 2 commits intoapache:2.0from
vvcephei:2.0-memory-leaks
Dec 4, 2018
Merged

KAFKA-7660: fix streams and Metrics memory leaks#5980
mjsax merged 2 commits intoapache:2.0from
vvcephei:2.0-memory-leaks

Conversation

@vvcephei
Copy link
Copy Markdown
Contributor

Backport two memory-leak fixes (#5974 and #5953) (see also 2.1: #5979)

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

In StreamsMetricsImpl, the parentSensors map was keeping references to Sensors after the sensors themselves had been removed.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
A heap dump provided by Patrik Kleindl in https://issues.apache.org/jira/browse/KAFKA-7660 identifies the childrenSensors map in Metrics as keeping references to sensors alive after they have been removed.

This PR fixes it and adds a test to be sure.

Reviewers: Jason Gustafson <jason@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
@vvcephei
Copy link
Copy Markdown
Contributor Author

@mjsax @guozhangwang , This is a further backport of a couple of small memory-leak fixes from trunk.

I'm not sure if you want to keep them as separate commits in the backport as well...

@vvcephei
Copy link
Copy Markdown
Contributor Author

@ijuma I'm not sure who to ask about this; asking you because you've weighed in on a few build configuration issues I've seen...

It looks like the JDK10 build may be misconfigured:

00:01:16.561 * What went wrong:
00:01:16.561 Task 'findbugsMain' not found in root project 'kafka-pr-jdk10-scala2.12'.

I'm wondering if we want to go ahead and bump 2.0 builds up to jdk11? I think we were only testing jdk10 provisionally until jdk11 got released.

@vvcephei vvcephei changed the title 2.0 memory leaks KAFKA-7660: fix streams and Metrics memory leaks Nov 30, 2018
@mjsax
Copy link
Copy Markdown
Member

mjsax commented Dec 1, 2018

Retest this please

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Dec 2, 2018

retest this please

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Dec 2, 2018

@vvcephei thanks for the heads-up. I disabled findBugs in the Java 10 PR build since findBugs doesn't work with Java 9+. We can't support Java 11 in the older branches without backporting a lot of changes and we don't want to do that. Please let me know if there are still problems.

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Dec 3, 2018

Thanks @ijuma !

@mjsax mjsax added the streams label Dec 4, 2018
@mjsax mjsax merged commit 985287a into apache:2.0 Dec 4, 2018
@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Dec 4, 2018

Thanks, @mjsax !

@vvcephei vvcephei deleted the 2.0-memory-leaks branch December 4, 2018 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants