Task1: Rendering notebooks

[grst]

Hi @scverse/core-devs,

I think we all agree that (in one form or the other) our tutorials will be based on jupyter notebooks.

But how do we render them? I see the following requirements

• They should be available all from one nice looking website
• There should be some sort of interactivity (i.e. the possibility to execute the notebooks online)
• Ideally, we have the option to optionally execute notebooks by the CI (but it should be also possible to use an executed .ipynb file, as some of the notebooks might be too computationally expensive)

Possible solutions:

• jupyterbook (@Zethson)
• sphinx with myst-nb (we use that in the cookiecutter template already)

For interactivity the notebooks can be linked to various online services like binder or google colab. For binder it's easier to declare dependencies as a yml file, but google colab has access to GPU which is particularly useful for scvi-tools.
See also https://sphinx-book-theme.readthedocs.io/en/stable/launch.html#launch-buttons-for-interactivity

There's also thebe which would allow to run code cells directly within sphinx. I didn't like their demo too much though, and I also don't see a clear advantage over opening binder/colab.

Hosting

If we go for sphinx, we could consider readthedocs. But probably building our own CI gives us more flexibility and resources for executing the notebooks and we can simply push sphinx/jupyterbook to github pages. The only downside is that it is hard to have multiple versions of the documentation (but not sure we need that for the tutorials).

WDYT?

[Zethson]

Thank you very much for writing this up.

They should be available all from one nice looking website

Just to ensure that we are on the same page. We are only talking about scverse CORE package tutorials here, right? Or do you think that scverse validated ecosystem packages should also be allowed to have a tutorial or two here?

There should be some sort of interactivity (i.e. the possibility to execute the notebooks online)

Yes, please. We should design everything with this in mind. I think that we'll need GPU support though, right @adamgayoso ?

If we go for sphinx, we could consider readthedocs. But probably building our own CI gives us more flexibility and resources for executing the notebooks and we can simply push sphinx/jupyterbook to github pages. The only downside is that it is hard to have multiple versions of the documentation (but not sure we need that for the tutorials).

I'd prefer having full control. Multiple versions of the tutorials are certainly useful, but I think that we want to encourage people to work with the latest versions at any point anyways. If we support multiple rendered versions of the tutorials we might also have to provide support for their code. I'd strongly want to avoid this.

jupyterbook (@Zethson)

My experiences with Jupyterbook are pretty good. As always, there are some things that you wish you could customize etc, but the out of the box setup is very good.

[grst]

Just to ensure that we are on the same page. We are only talking about scverse CORE package tutorials here, right? Or do you think that scverse validated ecosystem packages should also be allowed to have a tutorial or two here?

I think package specific tutorials should stay with their packages. However, IMO there is room for cross-package tutorials here that also use ecosystem tutorials (e.g. decoupler functional analysis combined with scanpy).

We could also link to/highlight ecosystem tutorials in the future. This would go hand in hand with what I wrote on Zulip about making the ecosystem "first-class".

[grst]

With regard to the flexibility of executing notebooks myst-nb and jupyterbook are pretty compareable. After all, jupyter-book uses myst-nb internally as far as I understood.

So it seems to come down to personal perference if we want to use jupyterbook or sphinx. One argument for sphinx is that we use it everywhere else. One argument for jupyterbook that Lukas already managed to build a pretty comprehensive book with it 🤷

[ivirshup]

So it seems to come down to personal perference if we want to use jupyterbook or sphinx.

Jupyterbook is just a configuration of sphinx right? But very configured to make something that looks like a book?

While I think what we want overlaps a lot with Jupyter-book, I don't think we're making a book. Instead, I think we will just use a bunch of the jupyter-book components.

[ivirshup]

Split out discussion on interactivity to #3

[ivirshup]

I would also like to point to this related discussion from the pangeo community: https://discourse.pangeo.io/t/statement-of-need-integrating-jupyterbook-and-jupyterhubs-via-ci/2705

Using the interactive environment for building

One cool idea from that space is using the environment where users can interact with the notebooks for building them.

This seems like it could be possible with binderbot and may even have actions available through project pythia

Open contribution

IMO there is room for cross-package tutorials here that also use ecosystem tutorials (e.g. decoupler functional analysis combined with scanpy).

We could also link to/highlight ecosystem tutorials in the future. This would go hand in hand with what I wrote on Zulip about making the ecosystem "first-class".

The model discussed for pangeo is very much open contribution. I like this, especially since it does not put maintenance of the tutorials strictly on the core team.

[adamgayoso]

I think jupyter book makes sense for this project (and yes I see it as a specific usage of sphinx)

And yes, we need colab for gpu support (but right now it's annoyingly on python 3.7, but hopefully moves to 3.8 soon)

[adamgayoso]

By the way, this quant econ book has really nice css, etc

https://python.quantecon.org/heavy_tails.html

https://github.com/QuantEcon/quantecon-book-theme

[ivirshup]

Quantecon definitely has some nice docs infrastructure. Though I think some of the docs might use older versions of it. I believe they also do a lot of work with jupyter book.

[adamgayoso]

Yeah it's basically as easy to try as installing the package and doing

sphinx:
    config:
        html_theme: quantecon_book_theme

[grst]

By the way, this quant econ book has really nice css, etc

Not sure I like this better than the default jupyter book theme... The mix of serif and non-serif fonts is ugly.

One cool idea from that space is using the environment where users can interact with the notebooks for building them.

This seems like a good idea to me! Also ensures that the notebooks actually run there and not only on github actions.

Any preference of (sphinx + jupyterbook theme) vs jupyter book? I feel with using sphinx directly we might have more flexibility in the long run, but jupyterbook does seem to have sensible defaults 🤷

[ivirshup]

By the way, this quant econ book has really nice css, etc

I like the header, but not really the text. It's a lot of white space.

---

Any preference of (sphinx + jupyterbook theme) vs jupyter book? I feel with using sphinx directly we might have more flexibility in the long run, but jupyterbook does seem to have sensible defaults

I'm just figuring out what the difference between "just jupyterbook" and sphinx is. Looks like this is summarized at: https://jupyterbook.org/en/stable/explain/sphinx.html. Main differences include:

How tables of contents are defined

jupyter-book centralizes where the table of contents is defined. I don't feel super strongly about this, but wonder if it will be a problem for "included tutorials".

_config.yaml instead of conf.py

I think we're going to want more configuration than jupyterbooks _config.yml. One example: I'd like to have analytics, but I'd rather use our current goatcounter rather than google analytics.

Practically, jupyterbook uses the config.yaml to generate a conf.py, so we could just start from that conf.py

cli interface

Basically just a different command to handle the conf.py. CLI is not really on my list of pain points about sphinx though.