Adding ARM arch to CI

[tylerjereddy]

The recently-announced free availability of ARM nodes for open source CI may eventually be of interest to us as we continue to expand platform / architecture support.

[tylerjereddy]

I set up a shippable ARMv8 CI build of MDAnalysis from my fork shippable.yml and you can check out the build console from the latest attempt--that will almost certainly fail because of dependency stuff for now.

I started something similar for NumPy--more details there.

[orbeckst]

This would be of interest for, e.g., MDAnalysis/GridDataFormats#51.

[tylerjereddy]

Results are looking a little better now. The job currently times out -- MDA has some pretty heavy dependencies without official ARMv8 wheels available. SciPy is really painful at 30 minute compile time -- you can see the other times in the linked build.

There are complications with PyPI and ARMv8 wheels (see our discussion with Cython team: cython/cython#2561); in practice we could just pin the versions maybe and host our own binaries for the time being; also caching the built wheels should be an option, but we need a successful build for the cache to upload to Amazon S3; maybe I disable the tests for one run to get the cache upload so we save build times moving forward for initial tests.

[tylerjereddy]

Ok, we're caching the binary dependencies as much as possible for pip installs and the test suite is now running on ARMv8 as visible in the the results.

In short, for Python 2.7:
5 failed, 7249 passed, 71 skipped, 1 xfailed, 36015 warnings in 1937.81 seconds

Expanding to include Python 3.7 (3.6 not really available for now) would require a bit more manual intervention to get all the dependencies cached in a separate folder (I did that for NumPy though, so can be done eventually...).

[tylerjereddy]

I also need to hook into their results summary so our tests stats are nicely displayed in their system, but that's fairly straightforward.. just takes time!

[tylerjereddy]

Note that our test suite is pretty slow on ARMv8 at > 30 minutes; we'd probably have to pin the versions of all our pip installs to prevent timeouts in the event that i.e., pip picks up a new SciPy or something and then we can't refresh the cache because we don't have enough total build time.

[tylerjereddy]

I've added Python 3.7 to the test matrix and will let the cache update this evening hopefully & then we'll be in a position to test two versions on somewhat reasonable time scales.

[tylerjereddy]

the nodes aren't behaving reliably at the moment so stalled for progress; looks like the caching was done, but the tests are basically hanging unpredictably, compared to running ok a few days ago.