Skip to content

Conversation

@Gargi-jais11
Copy link
Contributor

@Gargi-jais11 Gargi-jais11 commented Mar 25, 2025

What changes were proposed in this pull request?

As VolumeDensity is already covered by report sub command, we will show bytesMoved (bytesBalanced) in status command.

Besides, the meaning of VolumeDensity is clear to end user, but the real value indicating is not that straight forward. Use the estimated pending move size(HDDS-12437) will help user to clearly understand how much data size need to move. And this bytesMoved is also a straight forward value, indicating the work disk balancer has done.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12546

How was this patch tested?

Added unit test to test bytesToMoveMB, bytesMovedMB and calculateEstimatedTimeLeft to show reasonable values when bytesToMove and bytesMoved are less than 1MB.

As well as tested manually by running docker cluster locally.

bash-5.1$ ozone admin datanode diskbalancer status
Status result:
Datanode                            Status          Threshold(%)    BandwidthInMB   Threads      SuccessMove  FailureMove  BytesMoved(MB)  EstBytesToMove(MB) EstTimeLeft(min)
ozone-datanode-4.ozone_default      RUNNING         0.0002          10              5            5            3            7136            896             2              
ozone-datanode-3.ozone_default      RUNNING         0.0002          10              5            7            4            13131           3080            6              

Note: Estimated time left is calculated based on the estimated bytes to move and the configured disk bandwidth.

@Gargi-jais11 Gargi-jais11 marked this pull request as ready for review March 25, 2025 09:04
@Gargi-jais11 Gargi-jais11 requested a review from ChenSammi March 26, 2025 11:06
@Gargi-jais11 Gargi-jais11 requested a review from ChenSammi March 27, 2025 05:15
Gargi Jaiswal added 5 commits March 28, 2025 09:21
# Conflicts:
#	hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerInfo.java
#	hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerService.java
#	hadoop-hdds/interface-client/src/main/proto/hdds.proto
#	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerManager.java
#	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerStatus.java
#	hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerStatusSubcommand.java
Copy link
Contributor

@ChenSammi ChenSammi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Gargi-jais11 .

@ChenSammi ChenSammi merged commit b8bff8a into apache:HDDS-5713 Apr 3, 2025
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants