HDDS-13598. Send disk balancer command directly to datanode

[Gargi-jais11]

What changes were proposed in this pull request?

Support to direct send disk balancer command to DN(Client-DN) instead of Client-SCM-DN .
This JIRA covers:

Implementation of direct client-to-DN communication protocol

• CLI command refactoring and new command structure
• Datanode-side RPC server implementation
• Integration with existing DiskBalancerService
• Unit tests for new protocol
• Removal of SCM part which was needed to be done

Removal of rest SCM DiskBalancer code and modification of all UT and IT tests will be done as part of HDDS-13878 issue.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13598

How was this patch tested?

Green CI : https://github.com/Gargi-jais11/ozone/actions/runs/19130217959
New Unit Test TestDiskBalancerProtocolServer
Old Unit, Integration and testdiskbalancer.robot tests will be updated as part of HDDS-13878.

Below is cli output for direct client to DN communication:

bash-5.1$ ozone admin datanode diskbalancer report --in-service-datanodes
Report result:
Datanode                                           VolumeDensity
ozone-datanode-1.ozone_default                     7.385030119160252E-4
ozone-datanode-5.ozone_default                     3.2333757089136883E-6
ozone-datanode-2.ozone_default                     1.940025425348213E-6
ozone-datanode-4.ozone_default                     1.2933502835515975E-6
ozone-datanode-3.ozone_default                     0.0

bash-5.1$ ozone admin datanode diskbalancer report -d ozone-datanode-1,ozone-datanode-3,ozone-datanode-4
Report result:
Datanode                                           VolumeDensity
ozone-datanode-1.ozone_default                     7.385030119160252E-4
ozone-datanode-4.ozone_default                     1.2933502835515975E-6
ozone-datanode-3.ozone_default                     0.0

bash-5.1$ ozone admin datanode diskbalancer status --in-service-datanodes
Status result:
Datanode                            Status          Threshold(%)    BandwidthInMB   Threads      SuccessMove  FailureMove  BytesMoved(MB)  EstBytesToMove(MB) EstTimeLeft(min)
ozone-datanode-3.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              
ozone-datanode-2.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              
ozone-datanode-1.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              
ozone-datanode-5.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              
ozone-datanode-4.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              

Note: Estimated time left is calculated based on the estimated bytes to move and the configured disk bandwidth.
bash-5.1$ ozone admin datanode diskbalancer status -d ozone-datanode-1,ozone-datanode-3,ozone-datanode-4
Status result:
Datanode                            Status          Threshold(%)    BandwidthInMB   Threads      SuccessMove  FailureMove  BytesMoved(MB)  EstBytesToMove(MB) EstTimeLeft(min)
ozone-datanode-1.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              
ozone-datanode-3.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              
ozone-datanode-4.ozone_default      STOPPED         10.0000         10              5            0            0            0               0               0              

Note: Estimated time left is calculated based on the estimated bytes to move and the configured disk bandwidth.

start, stop and update command :

bash-5.1$ ozone admin datanode diskbalancer update -t 0.0001 -b 100 -d ozone-datanode-1,ozone-datanode-3,ozone-datanode-4
Updated DiskBalancer configuration on nodes: [ozone-datanode-1, ozone-datanode-3, ozone-datanode-4]

bash-5.1$ ozone admin datanode diskbalancer start --in-service-datanodes
Started DiskBalancer on all IN_SERVICE nodes.

bash-5.1$ ozone admin datanode diskbalancer stop --in-service-datanodes
Stopped DiskBalancer on all IN_SERVICE nodes.

bash-5.1$ ozone admin datanode diskbalancer start -d ozone-datanode-1,ozone-datanode-3,ozone-datanode-4
Started DiskBalancer on nodes: [ozone-datanode-1, ozone-datanode-3, ozone-datanode-4]

[Gargi-jais11]

@ChenSammi , @sumitagrawl , @szetszwo Could you please review this patch?

[ChenSammi]

@Gargi-jais11 , thanks for working on this, overall it looks good to me.

bash-5.1$ ozone admin datanode diskbalancer start -d ozone-datanode-1,ozone-datanode-3,ozone-datanode-4
Started DiskBalancer on all IN_SERVICE nodes.

Is the output a mistake?

[Gargi-jais11]

@Gargi-jais11 , thanks for working on this, overall it looks good to me.

bash-5.1$ ozone admin datanode diskbalancer start -d ozone-datanode-1,ozone-datanode-3,ozone-datanode-4
Started DiskBalancer on all IN_SERVICE nodes.

Is the output a mistake?

Yes, I think while pasting this happened. I will correct it here.

[ChenSammi]

The last patch LGTM + 1. Wait for the CI to pass.