Skip to content

MINOR: AbstractCoordinatorTest should close coordinator explicitly#10001

Merged
chia7712 merged 2 commits intoapache:trunkfrom
chia7712:MINOR-9999
Jan 30, 2021
Merged

MINOR: AbstractCoordinatorTest should close coordinator explicitly#10001
chia7712 merged 2 commits intoapache:trunkfrom
chia7712:MINOR-9999

Conversation

@chia7712
Copy link
Copy Markdown
Member

I noticed this issue when digging into some flaky by JVM profiler. AbstractCoordinatorTest does not close coordinator so it can cause a lot of idle heartbeat threads in the following tests.

"kafka-coordinator-heartbeat-thread | dummy-group" #239 daemon prio=5 os_prio=0 cpu=4.40ms elapsed=29.26s tid=0x00007f4798c34000 nid=0x11b6 in Object.wait()  [0x00007f471dbf5000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1398)
	- waiting to re-lock in wait() <0x000000008152b250> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

"kafka-coordinator-heartbeat-thread | dummy-group" #240 daemon prio=5 os_prio=0 cpu=4.15ms elapsed=29.16s tid=0x00007f4798c36800 nid=0x11b7 in Object.wait()  [0x00007f471d7f4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1398)
	- waiting to re-lock in wait() <0x000000008152e9c0> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

"kafka-coordinator-heartbeat-thread | dummy-group" #242 daemon prio=5 os_prio=0 cpu=0.23ms elapsed=29.04s tid=0x00007f4798c39000 nid=0x11b9 in Object.wait()  [0x00007f471d3f3000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at java.lang.Object.wait(java.base@11.0.9.1/Object.java:328)
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1355)
	- waiting to re-lock in wait() <0x00000000815107f8> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

"kafka-coordinator-heartbeat-thread | dummy-group" #244 daemon prio=5 os_prio=0 cpu=3.62ms elapsed=29.03s tid=0x00007f4798c3b000 nid=0x11bb in Object.wait()  [0x00007f471cff2000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1398)
	- waiting to re-lock in wait() <0x00000000815330b0> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

"kafka-coordinator-heartbeat-thread | dummy-group" #245 daemon prio=5 os_prio=0 cpu=4.09ms elapsed=28.93s tid=0x00007f4798c3d800 nid=0x11bc in Object.wait()  [0x00007f471cbf1000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1398)
	- waiting to re-lock in wait() <0x00000000815387e8> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

"kafka-coordinator-heartbeat-thread | dummy-group" #246 daemon prio=5 os_prio=0 cpu=4.14ms elapsed=28.83s tid=0x00007f4798c3f000 nid=0x11bd in Object.wait()  [0x00007f471c7f0000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1398)
	- waiting to re-lock in wait() <0x00000000815389d8> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

"kafka-coordinator-heartbeat-thread | dummy-group" #247 daemon prio=5 os_prio=0 cpu=4.08ms elapsed=28.72s tid=0x00007f4798c41800 nid=0x11be in Object.wait()  [0x00007f46e3ffe000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9.1/Native Method)
	- waiting on <no object reference available>
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1398)
	- waiting to re-lock in wait() <0x0000000081538bc8> (a org.apache.kafka.clients.consumer.internals.AbstractCoordinatorTest$DummyCoordinator)

I don't observe the relationship between this issue and flaky. However, it seems to me explicitly releasing idle resource is always a good pattern.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

Copy link
Copy Markdown
Contributor

@hachikuji hachikuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left a minor suggestion.

Just a couple minutes earlier and you could have gotten PR 10,000!

@chia7712
Copy link
Copy Markdown
Member Author

the flaky is traced by #9777

@chia7712 chia7712 merged commit 23de8cb into apache:trunk Jan 30, 2021
@chia7712 chia7712 deleted the MINOR-9999 branch March 25, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants