ARROW-17079: Show HTTP status code for unknown S3 errors #14019
Conversation
| S3ErrorToString(static_cast<Aws::S3::S3Errors>(error.GetErrorType())), " during ", | ||
| operation, " operation: ", error.GetMessage()); | ||
| S3ErrorToString(static_cast<Aws::S3::S3Errors>(error.GetErrorType())), | ||
| " (http status ", static_cast<int>(error.GetResponseCode()), ")", |
There was a problem hiding this comment.
Shouldn't we add this only if the AWS error is actually "unknown"?
There was a problem hiding this comment.
Yes you are right, that will be less confusing I think (instead of the user needing to juggle with two errors).
|
@pitrou This should be ready now :) |
pitrou
changed the title
pitrou
changed the title
|
Benchmark runs are scheduled for baseline = 3e40cd3 and contender = a5ecb0f. a5ecb0f is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
|
['Python', 'R'] benchmarks have high level of regressions. |
|
Benchmark runs are scheduled for baseline = 3e40cd3 and contender = a5ecb0f. a5ecb0f is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
This is the last change I propose to improve our S3 error message. For certain errors, unfortunately the AWS SDK is doing a poor job in propagating the error and just reports UNKNOWN (see https://github.com/aws/aws-sdk-cpp/blob/1614bce979a201ada1e3436358edb7bd1834b5d6/aws-cpp-sdk-core/source/client/AWSClient.cpp#L77), in these cases the HTTP status code can be an important source to find out what is going wrong (and is also reported by boto3). This has the downside of cluttering the error message a bit more, but in general this information will be very valuable to diagnose the problem. Given that we now have the API call and the HTTP status error, in general there is good documentation on the internet that helps diagnose the problem. Before: > When getting information for key 'test.csv' in bucket 'pcmoritz-test-bucket-arrow-errors': AWS Error UNKNOWN during HeadObject call: No response body. After: > When getting information for key 'test.csv' in bucket 'pcmoritz-test-bucket-arrow-errors': AWS Error UNKNOWN **(HTTP status 400)** during HeadObject call: No response body. Lead-authored-by: Philipp Moritz <pcmoritz@gmail.com> Co-authored-by: Antoine Pitrou <pitrou@free.fr> Signed-off-by: Antoine Pitrou <antoine@python.org>
3 participants
This is the last change I propose to improve our S3 error message.
For certain errors, unfortunately the AWS SDK is doing a poor job in propagating the error and just reports UNKNOWN (see https://github.com/aws/aws-sdk-cpp/blob/1614bce979a201ada1e3436358edb7bd1834b5d6/aws-cpp-sdk-core/source/client/AWSClient.cpp#L77), in these cases the HTTP status code can be an important source to find out what is going wrong (and is also reported by boto3).
This has the downside of cluttering the error message a bit more, but in general this information will be very valuable to diagnose the problem. Given that we now have the API call and the HTTP status error, in general there is good documentation on the internet that helps diagnose the problem.
Before:
After: