Skip to content

MINOR: Fix various memory leaks in tests#12959

Merged
cadonna merged 3 commits intoapache:trunkfrom
lucasbru:test_leaks
Dec 21, 2022
Merged

MINOR: Fix various memory leaks in tests#12959
cadonna merged 3 commits intoapache:trunkfrom
lucasbru:test_leaks

Conversation

@lucasbru
Copy link
Copy Markdown
Member

@lucasbru lucasbru commented Dec 6, 2022

Various tests in the streams park were leaking native memory.

Most tests were fixed by closing the corresponding rocksdb resource.

I tested that the corresponding leak is gone by using a previous rocksdb
release with finalizers and checking if the finalizers would be called at some
point.

Copy link
Copy Markdown
Member

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @lucasbru !

Here my feedback!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you close the state store in the test here but not in the other tests?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, that's a duplicate

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments above.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 152 to 154
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be enough to just optionsFacadeDbOptions.close();. You already verify that optionsFacadeDbOptions.close(); calls mockedDbOptions.close(); in the try-block. No need to verify it again.
Alternatively, you could add close to the list of ignored methods (ignoreMethods) and verify as you did. However, you need to add verify(mockedDbOptions) after optionsFacadeDbOptions.close() otherwise nothing is verified.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just doing this so that easymock doesn't complain about close being called on the unterlying mock. I want to call close here to free up resources, and the best way here is to tell easymock what to expect on the mocks.

I found an alternative way to do this via resetToNice which should be less confusing.

lucasbru and others added 3 commits December 19, 2022 21:02
Various tests in the streams park were leaking native
memory.

Most tests were fixed by closing the corresponding
rocksdb resource.
…/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapterTest.java

Co-authored-by: Bruno Cadonna <cadonna@apache.org>
Copy link
Copy Markdown
Member

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates, @lucasbru !

LGTM!

@cadonna
Copy link
Copy Markdown
Member

cadonna commented Dec 21, 2022

Build failures are not related:

Build / JDK 11 and Scala 2.13 / org.apache.kafka.clients.consumer.StickyAssignorTest.testLargeAssignmentAndGroupWithNonEqualSubscription()
Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSourceConnector
Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[1] Type=Raft-CoReside, Name=testDescribeQuorumReplicationSuccessful, MetadataVersion=3.4-IV1, Security=PLAINTEXT
Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[2] Type=Raft-Distributed, Name=testDescribeQuorumReplicationSuccessful, MetadataVersion=3.4-IV1, Security=PLAINTEXT
Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[5] Type=Raft-CoReside, Name=testDescribeQuorumReplicationSuccessful, MetadataVersion=3.4-IV1, Security=PLAINTEXT
Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[6] Type=Raft-Distributed, Name=testDescribeQuorumStatusSuccessful, MetadataVersion=3.4-IV1, Security=PLAINTEXT
Build / JDK 8 and Scala 2.12 / org.apache.kafka.tools.MetadataQuorumCommandTest.[1] Type=Raft-CoReside, Name=testDescribeQuorumStatusSuccessful, MetadataVersion=3.4-IV1, Security=PLAINTEXT
Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsWithCustomForwardingAdminIntegrationTest.testCreatePartitionsUseProvidedForwardingAdmin()
Build / JDK 17 and Scala 2.13 / kafka.api.ProducerIdExpirationTest.testProducerIdExpirationWithNoTransactions(String).quorum=zk
Build / JDK 17 and Scala 2.13 / kafka.server.DynamicBrokerReconfigurationTest.testTrustStoreAlter(String).quorum=kraft

@cadonna cadonna merged commit 26daa8d into apache:trunk Dec 21, 2022
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Jan 25, 2023
Various tests in the streams park were leaking native memory.

Most tests were fixed by closing the corresponding rocksdb resource.

I tested that the corresponding leak is gone by using a previous rocksdb
release with finalizers and checking if the finalizers would be called at some
point.

Reviewer: Bruno Cadonna <cadonna@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants