-
Notifications
You must be signed in to change notification settings - Fork 594
HDDS-12131. Overwrite an empty file with multipart-upload throws NPE #7739
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
adoroszlai
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @k5342 for reporting this bug and providing the fix.
For reference, this is where bucket info is set to null to prevent unnecessary update for empty file:
Lines 338 to 341 in 47525b2
| } else if (!isNamespaceUpdate) { | |
| // If no bucket size and Namespace changed, prevent from updating | |
| // bucket object. | |
| omBucketInfo = null; |
Let me know if more tests are suggested
I would like to suggest adding this simple robot test. I reproduced the NPE with it, and verified the fix.
diff --git hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
index e630fe6cda..22e26953f1 100644
--- hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
+++ hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
@@ -61,6 +61,12 @@ Test Multipart Upload With Adjusted Length
Perform Multipart Upload ${BUCKET} multipart/adjusted_length_${PREFIX} /tmp/part1 /tmp/part2
Verify Multipart Upload ${BUCKET} multipart/adjusted_length_${PREFIX} /tmp/part1 /tmp/part2
+Overwrite Empty File
+ Execute touch ${TEMP_DIR}/empty
+ Execute AWSS3Cli cp ${TEMP_DIR}/empty s3://${BUCKET}/empty_file_${PREFIX}
+ Perform Multipart Upload ${BUCKET} empty_file_${PREFIX} /tmp/part1 /tmp/part2
+ Verify Multipart Upload ${BUCKET} empty_file_${PREFIX} /tmp/part1 /tmp/part2
+
Test Multipart Upload
${uploadID} = Initiate MPU ${BUCKET} ${PREFIX}/multipartKey
${nextUploadID} = Initiate MPU ${BUCKET} ${PREFIX}/multipartKey…oad (apache#7739) (cherry picked from commit db059c6)
…oad (apache#7739) (cherry picked from commit db059c6)
What changes were proposed in this pull request?
S3MultipartUploadCompleteRequestWithFSO seems to throw NPE when an empty file is being overwritten by non-zero file because omBucketInfo allows null value (null value is passed when non-update needed). This patch fix this by checking omBucketInfo before use.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-12131
How was this patch tested?
I've added annotation for
@Nullable.Local compilation are passed and it'll be checked by CI. Let me know if more tests are suggested.