Skip to content

Conversation

@larsoner
Copy link
Member

Closes #3357.

Today is Sunday right @agramfort? :)

@larsoner larsoner added this to the 0.13 milestone Jun 29, 2016
@larsoner
Copy link
Member Author

From the first commit, which only touched circle.yml we get the html_dev-noplot:

https://1413-1301584-gh.circle-artifacts.com/0//home/ubuntu/mne-python/doc/_build/html/auto_tutorials/plot_visualize_raw.html

And from the second commit, which touched a couple of tutorials, we get the following run:

BUILD_DEV_HTML=1 sphinx-build -D plot_gallery=1 -D sphinx_gallery_conf.filename_pattern=\(plot_visualize_raw.py\|plot_background_filtering.py\) -b html -d _build/doctrees   . _build/html

Which produces:

https://1414-1301584-gh.circle-artifacts.com/0//home/ubuntu/mne-python/doc/_build/html/auto_tutorials/plot_background_filtering.html

https://1414-1301584-gh.circle-artifacts.com/0//home/ubuntu/mne-python/doc/_build/html/auto_tutorials/plot_visualize_raw.html

I like it, but I don't think we can merge this until the caching problem is solved. This build took an hour and 11 minutes, 57 minutes of which was downloading data (!).

@agramfort
Copy link
Member

LGTM

thx @Eric89GXL

@larsoner
Copy link
Member Author

Okay I'm going to rebase, hopefully it picks up the cache and takes less time now.

@larsoner
Copy link
Member Author

@jaeilepp looks like we still get:

restoring cache v5/os:linux/mne-tools/mne-python/dependency/pull/3365/1413/TvoMGrNbQpVtayCqXueI7A__.tar.gz
restoring home/ubuntu/.m2, home/ubuntu/.ivy2, home/ubuntu/.go_workspace, home/ubuntu/.gradle, home/ubuntu/miniconda, home/ubuntu/mne-tools.github.io

After this finishes successfully, I'll try pushing another commit. Maybe it needs a couple of tries to save properly.

@larsoner
Copy link
Member Author

Still appears to have failed:

restoring cache v5/os:linux/mne-tools/mne-python/dependency/pull/3365/1413/TvoMGrNbQpVtayCqXueI7A__.tar.gz
restoring home/ubuntu/.m2, home/ubuntu/.ivy2, home/ubuntu/.go_workspace, home/ubuntu/.gradle, home/ubuntu/miniconda, home/ubuntu/mne-tools.github.io

I don't understand why it would get the other directories. That's weird.

@jaeilepp
Copy link
Contributor

Yeah I'm trying to debug this by running minimal builds on my branch. Let's see if I can find something.

@jaeilepp
Copy link
Contributor

I'm wondering if our cache is just stuck in some weird state and it would fix itself if we just clear it once. I can't think of anything else right now.

@larsoner
Copy link
Member Author

Might as well try it

@jaeilepp
Copy link
Contributor

Done. Build without cache running now.

@larsoner
Copy link
Member Author

Well the save cache step looks like it saved the directories. We'll have to see how it does on the next build.

@larsoner
Copy link
Member Author

Argh, still only restored the ones from before:

https://circleci.com/gh/mne-tools/mne-python/1432

@larsoner
Copy link
Member Author

@jaeilepp
Copy link
Contributor

jaeilepp commented Jul 1, 2016

Argh, still only restored the ones from before:

I think how it works is that it uses the previous cache from the same branch. Only for new branches it uses the master's cache. I'll try with a new branch and see how it looks.

@jaeilepp
Copy link
Contributor

jaeilepp commented Jul 1, 2016

And looks like for PRs it is using the master of the fork, so basically we'll see if clearing the cache worked when we merge something.

@jaeilepp
Copy link
Contributor

jaeilepp commented Jul 1, 2016

So the problem with this might be that every PR is using the cache of the user instead of upstream master. In that case, I think this would be reasonable if we could somehow only download the data that is being used.

@larsoner
Copy link
Member Author

larsoner commented Jul 1, 2016

Actually, in that case, we just won't auto-download anything. Each example when it runs will trigger a download if necessary. Let's try it.

@jaeilepp
Copy link
Contributor

jaeilepp commented Jul 1, 2016

But the caching happens before the testing. So if we download at testing stage nothing gets cached. So at least for the master, the downloads should stay.

@larsoner
Copy link
Member Author

larsoner commented Jul 1, 2016

So if we download at testing stage nothing gets cached. So at least for the master, the downloads should stay.

Yeah -- check the commit I pushed, this is what happens. For master builds it downloads during the first phase as before. For other builds, it does some (weak) introspection to see if it should get the sample dataset at least. In theory we could expand it to other datasets but this should get 90% of use cases hopefully.

@codecov-io
Copy link

Current coverage is 86.67%

Merging #3365 into master will increase coverage by 34.43%

@@             master      #3365   diff @@
==========================================
  Files           335        335            
  Lines         57628      57628            
  Methods           0          0            
  Messages          0          0            
  Branches       8721       8781      +60   
==========================================
+ Hits          30105      49948   +19843   
+ Misses        25694       5004   -20690   
- Partials       1829       2676     +847   

Sunburst

Powered by Codecov. Last updated by 1a641d7...1ebd8ad

@larsoner larsoner changed the title ENH: Build relevant examples MRG: Build relevant examples Jul 1, 2016
@larsoner
Copy link
Member Author

larsoner commented Jul 1, 2016

Hooray, it worked!

https://circleci.com/gh/mne-tools/mne-python/1446#artifacts

@jaeilepp feel free to merge if you're happy.

@larsoner
Copy link
Member Author

larsoner commented Jul 1, 2016

(I killed the Travis build manually since the last commit only changed Circle, look one commit back to see Travis passing)

@agramfort agramfort merged commit 52a3ca0 into mne-tools:master Jul 1, 2016
@agramfort
Copy link
Member

thx @Eric89GXL

jona-sassenhagen added a commit to jona-sassenhagen/mne-python that referenced this pull request Jul 5, 2016
* 'master' of git://github.com/mne-tools/mne-python: (48 commits)
  FIX: Flake
  fix pep8
  [MRG] FIX Topographic plotting for KIT-UMD data (mne-tools#3349)
  QUICKFIX: logging test
  Error message for simulate evoked. (mne-tools#3372)
  Warnings for big bems. (mne-tools#3374)
  Add test for baseline correction.
  Use apply_baseline in read_evokeds.
  Makes (None, 0) the default value baseline in apply_baseline.
  [FIX] Adds apply_baseline to Evoked
  fix component inds in ica tutorial (mne-tools#3379)
  FIX: Xdawn with shuffled epochs (mne-tools#3373)
  FIX consistency:  Epochs.load_data() should always return self (mne-tools#3376)
  MRG: Build relevant examples (mne-tools#3365)
  [MRG] set vmin, vmax after smoothing in plot_epochs_image (mne-tools#3360)
  [FIX] Attempt to fix circle.
  fixing a bg plotting bug for topo plots
  Modified test and tutorial.
  [MRG] convert surface using python. (mne-tools#3273)
  [MRG] raw.plot_psd() with flat channel (mne-tools#3342)
  ...
@larsoner larsoner deleted the circlex branch November 28, 2016 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants