Caching things in Travis build

[richardjgowers]

As originally spotted by @utkbansal, installing our dependencies on travis can take 6-7 minutes, we could use the caching facility on travis to reduce this.

Our build is a little complicated as we're using conda, but it looks like scikit-learn have done this:
scikit-learn/scikit-learn#5455

@utkbansal is this something that you could look at?

[utkbansal]

I'll try to read more about this and get started on the coming weekend.

[utkbansal]

@richardjgowers Looks like you will have to turn on this feature in TravisCI, it's not available by default.

To use the caching feature, in your repository settings, set Build branch updates to ON.

[kain88-de]

Caches should be activated.

[orbeckst]

I think we need to

• cache pip (basically, ~/.pip/cache); I think we also need to make sure that we use pip install --upgrade so that we do pull in new versions of dependencies and not inadvertently keep stale packages around
• cache conda

Perhaps something like

cache: 
   pip: true
   directories:
       - $HOME/miniconda

will work?

[utkbansal]

@kain88-de @orbeckst @richardjgowers @jbarnoud Why are we dependent on astropy's ci helpers? Why don't we have our own?

We might need to modify them to work with caching.

[kain88-de]

They are very responsive and so far did everything we needed them to do. They are also used by a lot of other projects so we can assume they do stuff correct. If you have problems with the script you can run from a fork temporarily and then we fix it upstream. They might be also interested in the caching.

[orbeckst]

Sort of related to the problem of keeping our run time on travis under 50 mins: We could split the tests in analysis and core using the build matrix (which was suggested in #286 a long time ago). However, the problem has been to maintain coverage. Assuming that #1369 works and we can merge the coverage reports we could now make use of Travis' new feature Build Stages: As I understand it, we can run our matrix as one stage and then run another stage that combines the coverage reports (if the first stage is successful). We could use a GitHub repo or an S3 bucket as temporary storage for coverage reports; I don't know if Travis allows us to keep build artifacts from earlier stages.

[kain88-de]

One thing. Since we now run different python versions on travis the caching from help us. Dependencies have to be downloaded again for each new python version. Right now I only see general improvements in conda to reduce our installation times. Or alternatively reducing the number of dependencies we have.

[orbeckst]

This looks as if we decided not to pursue caching at this time.

I will close and if anyone is interested, please reopen.