Skip to content

KAFKA-14595 Move ReassignPartitionsCommand to java#13247

Merged
jolshan merged 106 commits intoapache:trunkfrom
nizhikov:KAFKA-14595
Nov 1, 2023
Merged

KAFKA-14595 Move ReassignPartitionsCommand to java#13247
jolshan merged 106 commits intoapache:trunkfrom
nizhikov:KAFKA-14595

Conversation

@nizhikov
Copy link
Copy Markdown
Contributor

This PR contains changes required to move PartitionReassignmentState class to java code.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@nizhikov nizhikov changed the title [WIP] KAFKA-14595 Move PartitionReassignmentState to java [WIP] KAFKA-14595 Move value object of ReassignPartitionsCommand to java Feb 14, 2023
@nizhikov
Copy link
Copy Markdown
Contributor Author

@mimaison

This PR is a first part of moving ReassignPartitionsCommand to java and tools module.
Can you, please, take a look?

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Feb 15, 2023

Is server-common the right module? What requires access to these classes aside from tools?

@nizhikov
Copy link
Copy Markdown
Contributor Author

@ijuma

Is server-common the right module?

When move ReassignPartitionsCommand finished all classes must land in tools module.
So the answer is No - server-common in not right module.

But, I propose to move with small steps and split moving in several changes.
To keep changes reviewable.
As a first step I propose to move case classes from ReassignPartitionsCommand to java code.

It seems new java classes can be keep in core module so I placed them into it.

@nizhikov nizhikov force-pushed the KAFKA-14595 branch 2 times, most recently from 3aa7274 to 77a363b Compare February 15, 2023 07:39
@nizhikov nizhikov changed the title [WIP] KAFKA-14595 Move value object of ReassignPartitionsCommand to java KAFKA-14595 Move value objects of ReassignPartitionsCommand to java Feb 15, 2023
@ijuma
Copy link
Copy Markdown
Member

ijuma commented Feb 15, 2023

If you want to do it in small steps, one way is that you introduce the new classes, but you do not update the command to use them. That way you can put them in the right destination from the start. In any case, I'll leave it to @mimaison to say how he'd prefer it.

@nizhikov
Copy link
Copy Markdown
Contributor Author

nizhikov commented Feb 15, 2023

Hello @ijuma

If you want to do it in small steps

It's more about simplifing review then my personal preferences :)

For now, I introduced java classes from ReassignPartitionsCommand in core module.
It seems we can review and merge current changes and then move to the actual command scala -> java transformation.

@nizhikov
Copy link
Copy Markdown
Contributor Author

@mimaison Please, share your feedback. Are you ready to review and merge current changes (only case classes and options moved to java code) or should I continue work and rewrite the whole command in Java?

@mimaison
Copy link
Copy Markdown
Member

mimaison commented Feb 20, 2023

I think we should directly move the classes to the tools module instead of temporarily keeping them in core. Since it's one of the large tools, it's ok to first create the new classes in tools and rewrite the tool in a follow up PR as Ismael suggested.

@nizhikov
Copy link
Copy Markdown
Contributor Author

nizhikov commented Feb 20, 2023

@mimaison

I think we should directly move the classes to the tools module instead of temporarily keeping them in core

Are you suggesting to perform all moving in one PR?
Or you have some plan how to performa moving in several steps? If yes, please, share some guidelines on how should I implement it.

@mimaison
Copy link
Copy Markdown
Member

Are you suggesting to perform all moving in one PR? Or you have some plan how to performa moving in several steps? If yes, please, share some guidelines on how should I implement it.

We can create the new Java classes in tools and not touch the existing Scala code. Then in a second time, we can rewrite the tool in Java using the new classes and delete all the old Scala code.

@nizhikov
Copy link
Copy Markdown
Contributor Author

We can create the new Java classes in tools and not touch the existing Scala code

Thanks for feedback.

OK. I will move new classes to tools and revert scala code changes made in this PR.

@nizhikov
Copy link
Copy Markdown
Contributor Author

@mimaison New classes moved to tools.

Please, share your opinion - should I continue work on this task and move task logic to java?
Or we will merge current changes and then move logic in separate PR?

@github-actions
Copy link
Copy Markdown

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)
If this PR is no longer valid or desired, please feel free to close it. If no activity occurrs in the next 30 days, it will be automatically closed.

@github-actions github-actions Bot added the stale Stale PRs label Jun 10, 2023
@mimaison mimaison added tools and removed stale Stale PRs labels Jun 22, 2023
Copy link
Copy Markdown
Contributor

@fvaleri fvaleri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just a few nits.

@nizhikov
Copy link
Copy Markdown
Contributor Author

CI looks OK

@mimaison
Copy link
Copy Markdown
Member

@jolshan Before merging you probably want to run the ReassignPartitionsTest and ThrottlingTest system tests to verify the command still works as expected.

@nizhikov
Copy link
Copy Markdown
Contributor Author

nizhikov commented Oct 20, 2023

@mimaison Thanks for the hint. I will run system tests, also.

@nizhikov
Copy link
Copy Markdown
Contributor Author

reassign_partitions_test.py is OK. Will check second in a minute.

$ git rev-parse HEAD
53e767a75f2d48c4e5419fdf83033f881f21771e

docker exec ducker01 bash -c "cd /opt/kafka-dev && ducktape --cluster-file /opt/kafka-dev/tests/docker/build/cluster.json  ./tests/kafkatest/tests/core/reassign_partitions_test.py "
/usr/local/lib/python3.9/dist-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
[INFO:2023-10-20 09:36:39,559]: starting test run with session id 2023-10-20--001...
[INFO:2023-10-20 09:36:39,559]: running 8 tests...
[INFO:2023-10-20 09:36:39,560]: Triggering test 1 of 8...
[INFO:2023-10-20 09:36:39,572]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': False, 'reassign_from_offset_zero': False, 'metadata_quorum': 'ISOLATED_KRAFT'}}
[INFO:2023-10-20 09:36:39,576]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: on run 1/1
[INFO:2023-10-20 09:36:39,578]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Setting up...
[INFO:2023-10-20 09:36:39,578]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Running...
[INFO:2023-10-20 09:39:03,424]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Tearing down...
[INFO:2023-10-20 09:39:27,516]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: PASS
[INFO:2023-10-20 09:39:27,517]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Data: None
[INFO:2023-10-20 09:39:27,534]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:39:27,534]: Triggering test 2 of 8...
[INFO:2023-10-20 09:39:27,545]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': False, 'reassign_from_offset_zero': False, 'metadata_quorum': 'ZK'}}
[INFO:2023-10-20 09:39:27,551]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK: on run 1/1
[INFO:2023-10-20 09:39:27,553]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK: Setting up...
[INFO:2023-10-20 09:39:31,390]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK: Running...
[INFO:2023-10-20 09:41:44,682]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK: Tearing down...
[INFO:2023-10-20 09:42:20,942]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK: PASS
[INFO:2023-10-20 09:42:20,944]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK: Data: None
[INFO:2023-10-20 09:42:20,962]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:42:20,963]: Triggering test 3 of 8...
[INFO:2023-10-20 09:42:20,983]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': False, 'reassign_from_offset_zero': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
[INFO:2023-10-20 09:42:20,987]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: on run 1/1
[INFO:2023-10-20 09:42:20,989]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Setting up...
[INFO:2023-10-20 09:42:20,989]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Running...
[INFO:2023-10-20 09:45:01,375]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Tearing down...
[INFO:2023-10-20 09:45:24,418]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: PASS
[WARNING - 2023-10-20 09:45:24,419 - runner_client - log - lineno:294]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Test requested 8 nodes, used only 7
[WARNING:2023-10-20 09:45:24,419]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Test requested 8 nodes, used only 7
[INFO:2023-10-20 09:45:24,420]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Data: None
[INFO:2023-10-20 09:45:24,441]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:45:24,441]: Triggering test 4 of 8...
[INFO:2023-10-20 09:45:24,456]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': False, 'reassign_from_offset_zero': True, 'metadata_quorum': 'ZK'}}
[INFO:2023-10-20 09:45:24,461]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: on run 1/1
[INFO:2023-10-20 09:45:24,464]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: Setting up...
[INFO:2023-10-20 09:45:28,235]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: Running...
[INFO:2023-10-20 09:48:01,185]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: Tearing down...
[INFO:2023-10-20 09:48:36,870]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: PASS
[WARNING - 2023-10-20 09:48:36,871 - runner_client - log - lineno:294]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: Test requested 8 nodes, used only 7
[WARNING:2023-10-20 09:48:36,871]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: Test requested 8 nodes, used only 7
[INFO:2023-10-20 09:48:36,872]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK: Data: None
[INFO:2023-10-20 09:48:36,887]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:48:36,888]: Triggering test 5 of 8...
[INFO:2023-10-20 09:48:36,902]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': True, 'reassign_from_offset_zero': False, 'metadata_quorum': 'ISOLATED_KRAFT'}}
[INFO:2023-10-20 09:48:36,907]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: on run 1/1
[INFO:2023-10-20 09:48:36,909]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Setting up...
[INFO:2023-10-20 09:48:36,911]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Running...
[INFO:2023-10-20 09:51:20,336]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Tearing down...
[INFO:2023-10-20 09:51:44,207]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: PASS
[INFO:2023-10-20 09:51:44,208]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT: Data: None
[INFO:2023-10-20 09:51:44,226]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:51:44,227]: Triggering test 6 of 8...
[INFO:2023-10-20 09:51:44,240]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': True, 'reassign_from_offset_zero': False, 'metadata_quorum': 'ZK'}}
[INFO:2023-10-20 09:51:44,244]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK: on run 1/1
[INFO:2023-10-20 09:51:44,247]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK: Setting up...
[INFO:2023-10-20 09:51:47,848]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK: Running...
[INFO:2023-10-20 09:54:10,056]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK: Tearing down...
[INFO:2023-10-20 09:54:46,389]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK: PASS
[INFO:2023-10-20 09:54:46,389]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK: Data: None
[INFO:2023-10-20 09:54:46,412]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:54:46,413]: Triggering test 7 of 8...
[INFO:2023-10-20 09:54:46,429]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': True, 'reassign_from_offset_zero': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
[INFO:2023-10-20 09:54:46,436]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: on run 1/1
[INFO:2023-10-20 09:54:46,439]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Setting up...
[INFO:2023-10-20 09:54:46,440]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Running...
[INFO:2023-10-20 09:57:44,797]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Tearing down...
[INFO:2023-10-20 09:58:09,838]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: PASS
[WARNING - 2023-10-20 09:58:09,838 - runner_client - log - lineno:294]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Test requested 8 nodes, used only 7
[WARNING:2023-10-20 09:58:09,838]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Test requested 8 nodes, used only 7
[INFO:2023-10-20 09:58:09,839]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT: Data: None
[INFO:2023-10-20 09:58:09,853]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 09:58:09,853]: Triggering test 8 of 8...
[INFO:2023-10-20 09:58:09,864]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'reassign_partitions_test.py', 'cls_name': 'ReassignPartitionsTest', 'method_name': 'test_reassign_partitions', 'injected_args': {'bounce_brokers': True, 'reassign_from_offset_zero': True, 'metadata_quorum': 'ZK'}}
[INFO:2023-10-20 09:58:09,867]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: on run 1/1
[INFO:2023-10-20 09:58:09,869]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: Setting up...
[INFO:2023-10-20 09:58:13,587]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: Running...
[INFO:2023-10-20 09:59:58,469]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: Tearing down...
[INFO:2023-10-20 10:00:36,352]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: PASS
[WARNING - 2023-10-20 10:00:36,352 - runner_client - log - lineno:294]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: Test requested 8 nodes, used only 7
[WARNING:2023-10-20 10:00:36,353]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: Test requested 8 nodes, used only 7
[INFO:2023-10-20 10:00:36,353]: RunnerClient: kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK: Data: None
================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.11.4
session_id:       2023-10-20--001
run time:         23 minutes 56.810 seconds
tests run:        8
passed:           8
flaky:            0
failed:           0
ignored:          0
================================================================================
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT
status:     PASS
run time:   2 minutes 47.940 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=False.metadata_quorum=ZK
status:     PASS
run time:   2 minutes 53.391 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT
status:     PASS
run time:   3 minutes 3.432 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=False.reassign_from_offset_zero=True.metadata_quorum=ZK
status:     PASS
run time:   3 minutes 12.410 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ISOLATED_KRAFT
status:     PASS
run time:   3 minutes 7.300 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=False.metadata_quorum=ZK
status:     PASS
run time:   3 minutes 2.145 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ISOLATED_KRAFT
status:     PASS
run time:   3 minutes 23.402 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.core.reassign_partitions_test.ReassignPartitionsTest.test_reassign_partitions.bounce_brokers=True.reassign_from_offset_zero=True.metadata_quorum=ZK
status:     PASS
run time:   2 minutes 26.485 seconds
--------------------------------------------------------------------------------

@nizhikov
Copy link
Copy Markdown
Contributor Author

throttling_test.py OK too:

[INFO:2023-10-20 10:03:55,974]: starting test run with session id 2023-10-20--002...
[INFO:2023-10-20 10:03:55,974]: running 2 tests...
[INFO:2023-10-20 10:03:55,974]: Triggering test 1 of 2...
[INFO:2023-10-20 10:03:55,988]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'throttling_test.py', 'cls_name': 'ThrottlingTest', 'method_name': 'test_throttled_reassignment', 'injected_args': {'bounce_brokers': False}}
[INFO:2023-10-20 10:03:55,991]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=False: on run 1/1
[INFO:2023-10-20 10:03:55,993]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=False: Setting up...
[INFO:2023-10-20 10:03:59,653]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=False: Running...
[INFO:2023-10-20 10:09:48,461]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=False: Tearing down...
[INFO:2023-10-20 10:11:00,903]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=False: PASS
[INFO:2023-10-20 10:11:00,904]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=False: Data: None
[INFO:2023-10-20 10:11:00,932]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO:2023-10-20 10:11:00,933]: Triggering test 2 of 2...
[INFO:2023-10-20 10:11:00,949]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 'throttling_test.py', 'cls_name': 'ThrottlingTest', 'method_name': 'test_throttled_reassignment', 'injected_args': {'bounce_brokers': True}}
[INFO:2023-10-20 10:11:00,952]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=True: on run 1/1
[INFO:2023-10-20 10:11:00,954]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=True: Setting up...
[INFO:2023-10-20 10:11:04,805]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=True: Running...
[INFO:2023-10-20 10:18:04,927]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=True: Tearing down...
[INFO:2023-10-20 10:19:12,329]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=True: PASS
[INFO:2023-10-20 10:19:12,330]: RunnerClient: kafkatest.tests.core.throttling_test.ThrottlingTest.test_throttled_reassignment.bounce_brokers=True: Data: None

@nizhikov
Copy link
Copy Markdown
Contributor Author

Note, this PR and #14588 are mutual exclusive.
After merging one, other must be merged with trunk

@nizhikov
Copy link
Copy Markdown
Contributor Author

@mimaison
@jolshan

Is there anything a have to do to finish this PR?

Copy link
Copy Markdown
Member

@jolshan jolshan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I will give Mickael a day or so for any further comments.

@nizhikov
Copy link
Copy Markdown
Contributor Author

@jolshan Looks like we good to go. Are you ready to merge this PR?

@nizhikov
Copy link
Copy Markdown
Contributor Author

Hello @mimaison
Can you, please, share your feeback for PR?
Are we ready to merge this?

@jolshan jolshan merged commit 76b1b50 into apache:trunk Nov 1, 2023
@nizhikov
Copy link
Copy Markdown
Contributor Author

nizhikov commented Nov 1, 2023

Thanks all for the help, review and merge!

k-wall pushed a commit to k-wall/kafka that referenced this pull request Nov 21, 2023
This PR is part of apache#13247
It contains changes to rewrite single test in java.
Intention is reduce changes in parent PR.

Reviewers: Luke Chen <showuon@gmail.com>, Taras Ledkov <tledkov@apache.org>
k-wall pushed a commit to k-wall/kafka that referenced this pull request Nov 21, 2023
…he#14456)

This PR is part of apache#13247
It contains ReassignPartitionsIntegrationTest rewritten in java.
Goal of PR is reduce changes size in main PR.

Reviewers: Taras Ledkov  <tledkov@apache.org>, Justine Olshan <jolshan@confluent.io>
yyu1993 pushed a commit to yyu1993/kafka that referenced this pull request Feb 15, 2024
This PR contains changes required to move PartitionReassignmentState class to java code.

Reviewers: Mickael Maison <mickael.maison@gmail.com>, Justine Olshan <jolshan@confluent.io>, Federico Valeri <fedevaleri@gmail.com>, Taras Ledkov Taras Ledkov <tledkov@apache.org>, Kamal Chandraprakash<kamal.chandraprakash@gmail.com>,
AnatolyPopov pushed a commit to aiven/kafka that referenced this pull request Feb 16, 2024
This PR is part of apache#13247
It contains changes to rewrite single test in java.
Intention is reduce changes in parent PR.

Reviewers: Luke Chen <showuon@gmail.com>, Taras Ledkov <tledkov@apache.org>
AnatolyPopov pushed a commit to aiven/kafka that referenced this pull request Feb 16, 2024
…he#14456)

This PR is part of apache#13247
It contains ReassignPartitionsIntegrationTest rewritten in java.
Goal of PR is reduce changes size in main PR.

Reviewers: Taras Ledkov  <tledkov@apache.org>, Justine Olshan <jolshan@confluent.io>
AnatolyPopov pushed a commit to aiven/kafka that referenced this pull request Feb 16, 2024
This PR contains changes required to move PartitionReassignmentState class to java code.

Reviewers: Mickael Maison <mickael.maison@gmail.com>, Justine Olshan <jolshan@confluent.io>, Federico Valeri <fedevaleri@gmail.com>, Taras Ledkov Taras Ledkov <tledkov@apache.org>, Kamal Chandraprakash<kamal.chandraprakash@gmail.com>,
Cerchie pushed a commit to Cerchie/kafka that referenced this pull request Feb 22, 2024
This PR is part of apache#13247
It contains changes to rewrite single test in java.
Intention is reduce changes in parent PR.

Reviewers: Luke Chen <showuon@gmail.com>, Taras Ledkov <tledkov@apache.org>
*
* @return A result that is useful for testing.
*/
static VerifyAssignmentResult verifyAssignment(Admin adminClient,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nizhikov, I found many methods are static methods (which means default permission), our application uses the function of partition assignment(not script), and we are using the partition assignment in our app by scala API. but now we can't access these static methods.

verifyAssignment(org.apache.kafka.clients.admin.Admin, java.lang.String, java.lang.Boolean)' is not public in 'org.apache.kafka.tools.reassign.ReassignPartitionsCommand'. Cannot be accessed from outside package

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It's an expected side effect.

Methods of this class are not public API and can be changed without any restrictions.
I advice you to modify your code.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, Thank for your reply

@ray841
Copy link
Copy Markdown

ray841 commented Jan 25, 2025

This PR contains changes required to move the Partition Reassignment State class to Java code.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

chia7712 pushed a commit that referenced this pull request Jun 24, 2025
chia7712 pushed a commit that referenced this pull request Jun 24, 2025
chia7712 pushed a commit that referenced this pull request Jun 24, 2025
jiafu1115 pushed a commit to jiafu1115/kafka that referenced this pull request Jul 2, 2025
jiafu1115 pushed a commit to jiafu1115/kafka that referenced this pull request Jul 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants