HDDS-8182. Add volume and container choosing policy #4408
Conversation
|
@ChenSammi Could you help to review the PR? |
| DefaultContainerChoosingPolicy.class, | ||
| ContainerChoosingPolicy.class).newInstance(); | ||
| } catch (Exception e) { | ||
| throw new RuntimeException(e); |
There was a problem hiding this comment.
Please log error meg here.
|
|
||
| @Override | ||
| public ContainerData chooseContainer(OzoneContainer ozoneContainer, | ||
| HddsVolume hddsVolume, Set<Long> inProgressContainerIDs) { |
There was a problem hiding this comment.
I think we should turn this API into a batch API.
ozoneContainer.getController().getContainers(hddsVolume) will be a quite expensive operation once the container per volume reach a high number. Try to mostly leverage the result in one chooseContainer call, batch allocation or cache the container list of this volume for a while.
There was a problem hiding this comment.
Added a new parameter "targetSize" for batch retrieve of containers here.
|
|
||
| // Can not generate DiskBalancerTask if we have less than 2 results | ||
| if (volumes.size() <= 1) { | ||
| return null; |
There was a problem hiding this comment.
Please log a debug level msg for this case.
| return diskBalancerConfiguration; | ||
| } | ||
| } No newline at end of file | ||
| } |
There was a problem hiding this comment.
Please revert this unnecessary change in this and the following two files.
There was a problem hiding this comment.
The checkstyle will prompt the following warnings:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DiskBalancerStatus.java
1: File does not end with a newline.
| LoggerFactory.getLogger(DiskBalancerConfiguration.class); | ||
|
|
||
| public static final String HDDS_DATANODE_DISK_BALANCER_VOLUME_CHOOSING_POLICY | ||
| = "hdds.datanode.disk.balancer.volume.choosing.policy"; |
There was a problem hiding this comment.
Please use the @config annotation for these two properties.
|
LGTM, +1. |
2 participants
What changes were proposed in this pull request?
This ticket is to add ContainerChoosingPolicy and VolumeChoosingPolicy for DiskBalancer Service.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8182
How was this patch tested?
unit test.