[C++] SSL-related arrow-s3fs-test failures with aws-sdk-cpp 1.9.51

[asfimport]

See possible hang on AppVeyor:

https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/39683796/job/olkossi0bpq4k186?fullLog=true

Running locally on Ubuntu produces multiple failures and then a crash:
https://gist.github.com/pitrou/eccdffa2161f7257bead90c63c10037c

Reporter: Antoine Pitrou / @pitrou
Assignee: Antoine Pitrou / @pitrou

PRs and other links:

• GitHub Pull Request #10620
• GitHub Pull Request #11401

_{Note: This issue was originally created as ARROW-13134. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
cc @kou @xhochy

[asfimport]

Antoine Pitrou / @pitrou:
One set of errors comes from this:
aws/aws-sdk-cpp#1690

[asfimport]

Antoine Pitrou / @pitrou:
Also note the weird curl error when trying to resolve a bucket's region:

[----------] 3 tests from S3RegionResolutionTest
[ RUN      ] S3RegionResolutionTest.PublicBucket
../src/arrow/filesystem/s3fs_test.cc:304: Failure
Failed
'_error_or_value31.status()' failed with IOError: When resolving region for bucket 'ursa-labs-taxi-data': AWS Error [code 99]: curlCode: 27, Out of memory
../src/arrow/filesystem/s3fs.cc:655  ResolveRegionUncached(bucket)
[  FAILED  ] S3RegionResolutionTest.PublicBucket (8919 ms)

This error disappears when forcing the scheme to "http" instead of "https"...

[asfimport]

Antoine Pitrou / @pitrou:
I've filed aws/aws-sdk-cpp#1691 for the HTTPS issue but it may be a problem on our packaging side (conda-forge?).

[asfimport]

Uwe Korn / @xhochy:
aws-sdk-cpp 1.9.44 should be available in ~30min, so you could also give that a try then.

[asfimport]

Antoine Pitrou / @pitrou:
aws-sdk-cpp 1.9.45 didn't make things better on AppVeyor:

https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/39699068/job/gb42ufxrgfrttaco?fullLog=true#L175

[asfimport]

Antoine Pitrou / @pitrou:
Also shows the same symptoms locally as 1.9.39.

[asfimport]

Antoine Pitrou / @pitrou:
For now, we've pinned the AWS SDK to 1.8.x as a workaround (see #10620). This doesn't fix this issue, though.

[asfimport]

Antoine Pitrou / @pitrou:
1.9.50 will hopefully fix some of the issues.

[asfimport]

Uwe Korn / @xhochy:
Merged 1.9.51 on conda-forge, will be available in 60-90min

[asfimport]

Antoine Pitrou / @pitrou:
Ok, 1.9.51 does fix some of the issues. However, the SSL-related "curlCode: 27, Out of memory" error is still there.

[asfimport]

Antoine Pitrou / @pitrou:
I can't reproduce locally anymore, the latest conda packages work fine here, and even 1.9.51 works fine (perhaps other conda dependencies were upgrade which fixed the underlying problem?).

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 11401
#11401