Add an lhapdf_compatibility module for LHAPDF

[scarlehoff]

This branch is now an update on top of #1861 that adds the possibility of running fits using pdfflow instead of lhapdf (during the fit lhapdf is used only for the t0 covmat and that should be covered fine by pdfflow).

Instead of directly importing lhapdf, the lhapdf imports are contained in an lhapdf_compatibility module which, if lhapdf is installed will simply provide it normally and transparently so nothing changes.

if lhapdf is not installed, it uses a combination of lhadpf_management and pdfflow to do whatever is needed from lhapdf.

wrt to those two libraries, lhapdf_management is feature complete in that it does all the management part of LHAPDF (installing, updating, looking around the metadata of a PDF) but it does nothing with the PDF themselves.
pdfflow instead is a port of the interpolation and extrapolation parts of LHAPDF but it is not really being maintained anymore. For what we need in the fit is enough and python-only (so we can run fits normally) but I would not swap LHAPDF for this library.

However, by concentrating all that is needed from LHAPDF in a single module it will be easier to swap LHAPDF for some other library that might appear in the future. We can also more easily extend LHAPDF. This will be relevant for instance for the nuclear PDFs.

The way I've included the changes has a nice perk (/cc @alecandido) which is the fact that one should be able to extract data/fktables from validphys without pdfflow or lhapdf (meaning, tensorflow is not needed for that).

[scarlehoff]

This now could actually be merged (after #1861 of course since it relies in the pyproject.toml installation)

[RoyStegeman]

Thanks, I'll have a look later but for now - do you want to keep lhapdf_management external to nnpdf?

[alecandido]

It's great, thanks @scarlehoff!

I believe we might want to keep an optional dependency on LHAPDF as long as it will be used (so almost forever, from present perspective), to test the grids produced.
But this should not interfere with the fit, so the lhapdf_compatibility approach is great.

In particular: lhapdf_compatibility is greater for transitioning, but the moment we'd have another reliable replacement, whatever it is (if ever), we do not even need to keep it side-by-side with lhapdf.

[scarlehoff]

Thanks, I'll have a look later but for now - do you want to keep lhapdf_management external to nnpdf?

Yes.

I believe we might want to keep an optional dependency on LHAPDF as long as it will be used (so almost forever, from present perspective), to test the grids produced.

Yes, for now the dependency on LHAPDF is mandatory from conda (and so when using the official distribution of nnpdf). This PR just opens the possibility of not using it (i.e., in development environments).

In order to use this PR you need to install the code like pip install .[nolha]

[scarlehoff]

@RoyStegeman did you have a chance to look at this?

[github-actions]

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

• Fit Name: NNBOT-bfbfcb622-2024-01-30
• Fit Report: https://vp.nnpdf.science/ba2V_NA4Q6iU5-SvwQF3mw==
• Fit Data: https://data.nnpdf.science/fits/NNBOT-bfbfcb622-2024-01-30.tar.gz

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

[RoyStegeman]

Everything looks okay. I'll have to take a bit of a leap of faith that pdfflow behaves the same as lhapdf and implements all the relevant functions, but the passing of the tests is a good sign.

And since lhapdf is still the recommended backend to interact with PDFs it's not our responsibility if this breaks things (right??)

[scarlehoff]

And since lhapdf is still the recommended backend to interact with PDFs it's not our responsibility if this breaks things (right??)

Yes. The idea is to be able to develop without having to have an lhpdf installation. This is particularly relevant when testing different version of python, you then need to install lhapdf on each of them even if you are just checking that e.g. validphys is creating the thcovmat in the right order.

[RoyStegeman]

What about postfit (and some other minor scripts that still use classic lhapdf)?

[scarlehoff]

What about postfit (and some other minor scripts that still use classic lhapdf)?

postfit (and reweighting and some other) will need lhapdf.
But these are scripts that generate a final lhapdf-product so I'd say it is fair.

but the passing of the tests is a good sign.

The tests are always using the official lhapdf installation.

[RoyStegeman]

The tests are always using the official lhapdf installation.

Haha I realized my mistake

[alecandido]

🎉