Use example scripts as tests.

[paquiteau]

Hi there,
This PR is a proposal to tackle the current main weaknesses of Modopt:

1. The use of dummy test to validate coverage (especially for optimization algorithms)
2. Its lack of examples

The idea is too use examples/tutorials script as "real-world" test cases (or at least more real than checking if the FB algorithm is able to solve the inverse problem y = x with x positive) .

In essence the same file is usable by pytest to solve (1.) and sphinx-gallery to address (2.)

For now I have implemented the LASSO problem (from the simulation example of the original article ¹. Most of the algorithms paper have such toy examples, and reproducing them in the example is a good way of showing what ModOpt is capable, and help in the discoverability of its API (the lasso example produces 41% of coverage all by itself !).

A few remarks:

• Note that I am not talking about benchmarking, I think this should be addressed separatly with e.g. benchop
• This is still a proof of concept, and some ideas are still on the table:
  • Separating the example script into several code and test them sequentially
  • Test multiple algorithm in the same file, and produce a rough comparison on this toy example (e.g. all the flavour of the FB algorithm).
• I have not yet configured the sphinx-gallery stuff, stay tuned.

I will greatly appreciate your feedback on this to see if we should continue in this direction.

Footnotes

1. Tibshirani, 1996 (https://www.jstor.org/stable/2346178) ↩

[paquiteau]

A preview of the rendered example can be found here

https://perso.crans.org/comby/neurospin/modopt/example/auto_examples/index.html

[sfarrens]

Wow @paquiteau this is really fantastic and 100% in the spirit of what I hoped for ModOpt when I started the package. 👏

I opened some threads to just fix some typos and tweak the docstrings, but they are all suggestions you can click to commit.

I really like the direction this is going and I think this will hugely increase the understanding and appreciation of all the tools that are already in ModOpt. Some minor comments/suggestions:

• Even if this a toy example, it helps to visualise the data and reconstruction to demonstrate that things are working. So perhaps a final plot of BETA_TRUE compared to fb_fista.x_final.
• Another nice feature is to link any mention of specific modules/operators etc. to the corresponding API page. You can do this by writing e.g.:

  """
  :py:func:`Linear <modopt.opt.linear>`
  """

  in the doctstrings (markdown cell lines). This way users can easily find more information about the tools being demonstrated.
• Even if everything is well referenced to specific examples in papers etc., it might be worth adding a tiny bit of LaTeX maths to some cells to give the big picture of the problem being solved.

Anyway, I am completely happy for you to play around with this format and I think we can quickly converge to a standardised format for these examples.

[paquiteau]

Alright I expanded the example a little, here is a new render:
https://perso.crans.org/comby/neurospin/modopt/example/auto_examples/index.html

[sfarrens]

Hi @paquiteau, very sorry for the slow review. 🙏

Can you rebase/pull the latest changes to the develop branch so that I double check that everything looks OK in this PR before merging?

[paquiteau]

Hello, everything should be good to go :)

[paquiteau]

Ah! I merged a bit to quickly and removed the dependencies to matplotlib for the test ...