Fixed #5954 Uploading large files with S3 - Multipart Upload

[mderuijter]

Related Issues

• closes Issue uploading large files with an S3 backend #5954 : Issue uploading large files with an S3 backend

Pull Request Checklist

• Unit tests completed
• Integration tests: None
• Deployment requirements, SQL updates, Solr updates, etc.: None
• Documentation completed
• Merged latest from "develop" branch and resolved conflicts

[coveralls]

Coverage increased (+0.0003%) to 19.461% when pulling c3fabf8 on mderuijter:5954-multipart-upload-support into b46243d on IQSS:develop.

[pdurbin]

@mderuijter thanks for this pull request!

The code looks fine but where you planning to add anything to the guides? Do you need any help with this?

Also, I noticed a failure at https://jenkins.dataverse.org//job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6341/lastCompletedBuild//testReport/junit/edu.harvard.iq.dataverse.api/AccessIT/AccessIT/ but I'm pretty sure this is because your branch is a little behind the "develop" branch. If you merge the latest from "develop" into your branch, you'll pick up pull request #6334, which should fix that error.

[pdurbin]

In a comment I mentioned that some docs might be nice. 😄

[mderuijter]

@mderuijter thanks for this pull request!

The code looks fine but where you planning to add anything to the guides? Do you need any help with this?

Also, I noticed a failure at https://jenkins.dataverse.org//job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6341/lastCompletedBuild//testReport/junit/edu.harvard.iq.dataverse.api/AccessIT/AccessIT/ but I'm pretty sure this is because your branch is a little behind the "develop" branch. If you merge the latest from "develop" into your branch, you'll pick up pull request #6334, which should fix that error.

Thank you @pdurbin, I was wondering what documentation to add and where to put it. A little help would be nice indeed.

I've followed your instructions regarding the jenkins failure and it seems to have passed all the checks.

edit: I spoke to soon, some checks were not successful

[pdurbin]

@poikilotherm do you have the capacity to push some documentation to this branch or at least stub something out so that @mderuijter can fill in the details?

@mderuijter thanks for merging the latest from develop. A new test is failing on build 2 (SwordIT.testCreateDatasetPublishDestroy) but I've seen this from time to time and I think it's unrelated to your branch: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-6341/2/

[poikilotherm]

@pdurbin: sorry, busy with #5974 right now and profiling #6332 / #6052

[pdurbin]

@mderuijter I added a note in https://github.com/mderuijter/dataverse/pull/1 which you are welcome to merge (or not). 😄

I'm moving this to QA. Thanks again!

[kcondon]

Hi @mderuijter @pdurbin, to activate this feature one would need s3 storage enabled and upload through either the browser or api or this intended for api use only?

[pdurbin]

@kcondon the code change is in the S3 driver itself which is used by both the UI and the API.