Refactoring of S3AccessIO driver

[poikilotherm]

Hi @landreev

while digging into the driver to get things running for #4690 and #5068 it seems you wrote most of the storage driver for AWS S3. (At least you opened #3921 😉 )

In #5059 I started work to implement unit and integration testing for the driver. Are you interested to join?

[pdurbin]

@poikilotherm I see you're already working away on this over at poikilotherm@5251045 adding unit tests. Great! Thanks!

[poikilotherm]

Yes I am - but it is quite an amount of work... And many details of the implementation are unknown to me.

Example: in getMainFileKey() it is not obvious to me, why this method is not written sideeffect free. I hoped that @landreev could help me out with a lots of details about the driver... 😉

[poikilotherm]

Maybe @ferrys, @matthew-a-dunlap, @bsilverstein, @oscardssmith and other can look into this, too?

[oscardssmith]

Most of the changes I made were making StorageIO parametric on the type of DVObject stored. My sense in general is that this area was rather underdesigned, and the reason for most of these types of questions is because "It worked and we want to move on". What is the side effect by the way?

[poikilotherm]

What is the side effect by the way?

Both the instance vars key and bucketName get set inside, although key is set again with the return value of the function. This could be by design or unintended - hard to predict without comments and not being involved before... 😉

[poikilotherm]

Mentioning #5165

[poikilotherm]

Mentioning #5996 here...

[pdurbin]

We now have some tests for S3AccessIO:

• add S3 tests, LocalStack, MinIO #10044