Make EKO into a separate resource

[scarlehoff]

Closes #2081

It can be tested with theory_40001002

From an user perspective there should be no differences. When you only need the fktables / theory (i.e., for reports or fits, when doing check_theoryID) only the fktables will be downloaded.
If the eko needs to be downloaded as well (only for evolution), then it will be downloaded into the same folder as the theory (done in this way so that all the theories that have the eko inside work just the same).

There's a separate check_eko(thid) which is used by evolven3fit and returns the path to the eko.tar of the theory, checking first the theory.

At the moment the ekos range between the 1.5 and 3.5 GB while the whole set of fktables used for 4.0 is about 600 MB so this can have a huge impact e.g. when doing reports in your laptop when the PDF has been generated elsewhere.

[RoyStegeman]

The thing is dislike a bit about this is that as part of my workflow I will often (manually) run vp-setupfit before submitting many jobs to make sure that all the parallel jobs won't try to download the same thing. This PR means I'll have to be more careful about it and find some way to download the ekos as well.

[RoyStegeman]

Ah no I just see evolven3fit is not on top of reportengine so there is no way to do --no-net which I think is import to prevent the issue of accidentally trying to download the same thing many times.

[scarlehoff]

Ah no I just see evolven3fit is not on top of reportengine so there is no way to do --no-net which I think is import to prevent the issue of accidentally trying to download the same thing many times.

I can add a --no-net option to evolven3fit if you want.
However, would you like me to add (in that case) a check_eko option to setupfit so that it downloads also the eko? So we can consider setupfit as a "download and check everything that I will need for the fit". n3fit will not check the eko.

Putting evolven3fit on top of validphys would fix quite a few issues but I'm certainly not doing that (in the short term)

[RoyStegeman]

I can add a --no-net option to evolven3fit if you want.

Yes, thanks.

However, would you like me to add (in that case) a check_eko option to setupfit so that it downloads also the eko?

I was thinking about asking this, but if we want to do an MHOU regression test that will rely on vp-setupfit to compute the covmat. Not sure what the limit for the CI is, would it be fine if it just downloads the eko along with theoryid from the theory key and not theoryids from the theorycovmatconfig (or whatever that key is called)?

[scarlehoff]

would it be fine if it just downloads the eko along with theoryid from the theory key and not theoryids from the theorycovmatconfig (or whatever that key is called)?

Yes. I think it also makes sense since only the eko for the fit's theoryid is needed for setting up the fit.

[RoyStegeman]

Yes. I think it also makes sense since only the eko for the fit's theoryid is needed for setting up the fit.

Of course, but it immediatly adds 1.5GB more to the CI so I was just asking/wondering how much wiggle room we have.

Anyway, downloading the eko with vp-setupfit would be my preferred solution

[scarlehoff]

Done. Let's see what does the fitbot think.

[scarlehoff]

"ok, everything works ok but it would be nice to add a nice small printout. FML" :)

[github-actions]

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

• Fit Name: NNBOT-d0e4dd190-2024-05-23
• Fit Report wrt master: https://vp.nnpdf.science/Eq8FvmDrRPuW3vE8gOOnLA==
• Fit Report wrt latest stable reference: https://vp.nnpdf.science/HdDTVnizRW6EhwGJv3PtBw==
• Fit Data: https://data.nnpdf.science/fits/NNBOT-d0e4dd190-2024-05-23.tar.gz

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

[scarlehoff]

Can I merge this @RoyStegeman / are you happy with the setupfit solution or do you want to have a second look?