Test example notebooks on demand #1613
Conversation
| - name: Run example notebooks | ||
| run: | | ||
| cd example-notebooks | ||
| python -m pytest -v setup/ showcase/ cookbook/ openmm_rbfe/ networks/ --nbval-lax --nbval-cell-timeout=3000 -n auto --dist loadscope |
There was a problem hiding this comment.
if we add / reorganize the directory this test will pass in a sneaky way - do we need to be so specific here?
There was a problem hiding this comment.
That is how we test it upstream, it is a good question but without specifying a subset of notebooks, they all get tested. Which we might want to do. Looking at the PR where I set this up, I don't see anything like "don't test this notebook" so maybe here (and upstream) we should change it to test everything?
There was a problem hiding this comment.
hm, my understanding is that we inherently do want to test anything that is in the example notebooks repo. otherwise it shouldn't be there
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1613 +/- ##
==========================================
- Coverage 95.34% 92.89% -2.45%
==========================================
Files 183 183
Lines 15779 15779
==========================================
- Hits 15044 14658 -386
- Misses 735 1121 +386
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
No API break detected ✅ |
|
IDK how long this takes to run, but this should be added to the review checklist since we want to make sure no PR breaks out example notebooks |
atravitz
left a comment
atravitz
left a comment
There was a problem hiding this comment.
looks good! agree that it can be in the checklist, but as a discretionary manual (not required or auto-CI) check.
Checklist
newsentryDevelopers certificate of origin