Skip to content

Conversation

@ptlrs
Copy link
Contributor

@ptlrs ptlrs commented Apr 1, 2025

Please describe your PR in detail:

  • This PR hooks into the NonHATests framework to obtain a shared MiniOzoneCluster that can be used by other CLI tests
  • It adds integration tests for the ozone debug replicas verify checksums command

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12715

How was this patch tested?

CI: https://github.com/ptlrs/ozone/actions/runs/16375920556

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ptlrs for working on this.

Moving cluster creation to an extension reduces duplication, but:

  • test execution is still unnecessarily long, because each test class needs to start its own cluster
  • the specific implementation does not allow config to be customized if needed

Please check TestOzoneIntegrationNonHA and its parent classes for a better way to reuse a shared cluster for multiple test classes (HDDS-12183 and several later sub-tasks of HDDS-9000).

@errose28 errose28 added the tools Tools that helps with debugging label Apr 1, 2025
@ptlrs
Copy link
Contributor Author

ptlrs commented Apr 8, 2025

Thanks for the review @adoroszlai.
I agree, the cluster config here is limited to what is set in the extension.

With the custom extension, the cluster is created only once for both the test classes. The cluster is initialized once and stored as a static member. I had confirmed this by logging the creation of MiniOzoneCluster.

Is the goal of the new approach to consolidate all tests that can reuse the same config/cluster?

@ptlrs ptlrs force-pushed the HDDS-12715-common-integration-test-cluster-for-debug-and-repair-tools branch from 42555b8 to 68fa7bb Compare April 16, 2025 20:00
@ptlrs ptlrs requested a review from adoroszlai April 30, 2025 17:13
@ptlrs
Copy link
Contributor Author

ptlrs commented Apr 30, 2025

Hi @adoroszlai, the PR has been updated to use the existing TestOzoneIntegrationNonHA class. Could you please take another look.

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ptlrs for updating the patch.

I think the PR title also needs to be updated.

@ptlrs ptlrs force-pushed the HDDS-12715-common-integration-test-cluster-for-debug-and-repair-tools branch from 857aae0 to a88c216 Compare May 19, 2025 17:42
@ptlrs ptlrs changed the title HDDS-12715. create a common miniozone cluster for ozone debug tools tests HDDS-12715. add integration tests for debug-replicas-verify-checksums tool May 19, 2025
@ptlrs ptlrs force-pushed the HDDS-12715-common-integration-test-cluster-for-debug-and-repair-tools branch from a88c216 to 0674e6c Compare May 19, 2025 17:49
@ptlrs ptlrs requested a review from adoroszlai May 19, 2025 18:00
@ptlrs
Copy link
Contributor Author

ptlrs commented Jun 3, 2025

Hi @adoroszlai @dombizita, could you please review this updated PR when you get a chance? Thanks.

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ptlrs for updating the patch.

@ptlrs ptlrs requested a review from adoroszlai July 18, 2025 19:14
@ptlrs
Copy link
Contributor Author

ptlrs commented Jul 18, 2025

Hi @adoroszlai, the test has been updated. Could you please take another look.

@adoroszlai adoroszlai dismissed their stale review July 20, 2025 18:12

patch updated

@adoroszlai adoroszlai requested review from dombizita and errose28 July 20, 2025 18:13
Copy link
Contributor

@dombizita dombizita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating your patch @ptlrs, it looks good to me! Please update the jira title, so it'll match with the actual implementation. One more thing, there is also a --block-existence and --container-state flag for the ozone debug replicas verify tool, we have acceptance tests for it, do you think it'd a good idea to add tests for those in this patch? I know this PR is pending for a while, so I'm fine with this state as well.

@dombizita dombizita requested a review from Tejaskriya July 23, 2025 13:04
@ptlrs ptlrs changed the title HDDS-12715. add integration tests for debug-replicas-verify-checksums tool HDDS-12715. Add integration tests for debug-replicas-verify-checksums tool Aug 22, 2025
@ptlrs
Copy link
Contributor Author

ptlrs commented Aug 22, 2025

Thanks for the review @dombizita. Let's open a new task for adding the tests for the block-existance and container-state features. This PR is in continuation of the work that was done for the checksums tool. I am not familiar with the other two features, so a new task to handle it spearately would be better.

@dombizita
Copy link
Contributor

@ptlrs sounds good to me, thanks! Could you please merge latest master to this branch? It's been a month since the last time, once that's done I'll start CI and if all looks good we can merge!

…ntegration-test-cluster-for-debug-and-repair-tools

Change-Id: I8c4bffdd79009c1755e89719ac3e79d0490515d0
@dombizita dombizita marked this pull request as ready for review September 3, 2025 11:16
Copy link
Contributor

@sarvekshayr sarvekshayr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @ptlrs.

Tests for the remaining verification flags can either be added in a follow-up JIRA or skipped since we already have acceptance tests covering them.

Overall, this LGTM.

Copy link
Contributor

@Tejaskriya Tejaskriya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @ptlrs , LGTM

@Tejaskriya Tejaskriya merged commit 54ade0c into apache:master Sep 4, 2025
38 of 39 checks passed
@Tejaskriya
Copy link
Contributor

Thanks for the reviews @dombizita @sarvekshayr @adoroszlai , and the contribution @ptlrs

@ptlrs ptlrs deleted the HDDS-12715-common-integration-test-cluster-for-debug-and-repair-tools branch September 5, 2025 18:23
peterxcli pushed a commit to peterxcli/ozone that referenced this pull request Sep 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tools Tools that helps with debugging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants