HDDS-4524. Create freon test to measure closed container replication

[elek]

What changes were proposed in this pull request?

Create new Freon test for the closed container replication.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4524

How was this patch tested?

#Generate keys
ozone ockg -n 10
#Close container
ozone container close 1
#Execute freon
ozone freon cr

Containers supposed to be visible under /tmp/....

Note: this patch requires the changes in #1551

[adoroszlai]

Containers supposed to be visible under /tmp

I think this depends on hdds.datanode.dir setting. In ozone compose env the container was replicated by default to /data/hdds. Got it to replicate to /tmp with: ozone freon -Dhdds.datanode.dir=/tmp cr -t 1.

On another note, trying to run ozone freon cr the second time leads to:

[main] ERROR volume.MutableVolumeSet: Failed to parse the storage location: /tmp
java.io.IOException: Volume is in an INCONSISTENT state. Skipped loading volume: /tmp/hdds
  at org.apache.hadoop.ozone.container.common.volume.HddsVolume.initialize(HddsVolume.java:225)

Is this expected? Do you know a workaround for that (other than deleting /tmp/hdds)?

[jojochuang]

Overall looks good. Just some nits

[elek]

Is this expected? Do you know a workaround for that (other than deleting /tmp/hdds)?

I can implement an automatic delete, but I think it's more safe to live it for the users. It seems to be more safe to fail when the directory is not empty (harder to understand what was downloaded) and ask users to delete the dir.

But I am not sure, can be convinced.

[elek]

@jojochuang Thanks for the suggestions, I updated the branch with them.

[adoroszlai]

I can implement an automatic delete, but I think it's more safe to live it for the users. It seems to be more safe to fail when the directory is not empty (harder to understand what was downloaded) and ask users to delete the dir.

Agreed. Can you add a comment somewhere to remind users to delete the dir in case of this error?

[elek]

I can implement an automatic delete, but I think it's more safe to live it for the users. It seems to be more safe to fail when the directory is not empty (harder to understand what was downloaded) and ask users to delete the dir.

Agreed. Can you add a comment somewhere to remind users to delete the dir in case of this error?

Sure. I added a check to return with more meaningful error. (5aa2fcc)

[adoroszlai]

Thanks @elek for updating the patch.

[elek]

Merged. Thanks the review @adoroszlai and @jojochuang