Skip to content

Remove libNNPDF from python#1680

Merged
scarlehoff merged 9 commits into
masterfrom
unlink_libNNPDF_fromPython
Mar 9, 2023
Merged

Remove libNNPDF from python#1680
scarlehoff merged 9 commits into
masterfrom
unlink_libNNPDF_fromPython

Conversation

@scarlehoff
Copy link
Copy Markdown
Member

@scarlehoff scarlehoff commented Feb 24, 2023

This PR removes all libNNPDF imports from python. The compilation still happens (not planning to remove that in this PR) so one can still do import NNPDF if needed.

I've tested the examples in the validphys/examples folder and they seem to work ok.

The main thing this PR actually does is to remove the load functions which in the past were loading the full dataset (fktables, cuts, commondata) just to do things like getting the number of points because that's what C++ asked for. I've changed these for load_commondata functions which seem to be enough everywhere (the only information requested by the calling function was whatever was contained in the commondata). Since the commondata is now pure python, no C++ is needed anymore.

@scarlehoff scarlehoff changed the title Remove libNNPDF from python [WIP] Remove libNNPDF from python Feb 24, 2023
@scarlehoff scarlehoff added the run-fit-bot Starts fit bot from a PR. label Feb 25, 2023
@github-actions
Copy link
Copy Markdown

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

@scarlehoff scarlehoff removed the run-fit-bot Starts fit bot from a PR. label Feb 27, 2023
@scarlehoff scarlehoff added the run-fit-bot Starts fit bot from a PR. label Feb 27, 2023
@scarlehoff scarlehoff marked this pull request as ready for review February 27, 2023 11:57
@scarlehoff
Copy link
Copy Markdown
Member Author

The only thing left in this PR is to add some tests to substitute the ones I've removed (because the test was basically a cross-check with the C++ version). I guess make them into regression tests is the right thing to do (because at the end of the day it was what they were). But suggestions welcome.

@scarlehoff scarlehoff changed the title [WIP] Remove libNNPDF from python Remove libNNPDF from python Feb 27, 2023
@scarlehoff
Copy link
Copy Markdown
Member Author

Special thanks to @comane and @andreab1997 who made this possible with their work on closure tests and th covmat!

@Zaharid
Copy link
Copy Markdown
Contributor

Zaharid commented Feb 27, 2023

Thanks for this. The diff looks nice and obvious, which is good.

@comane maybe you would like to poke around this a little bit (e.g. try to see if there is something that obviously crashes).

Someone should go over the docs as well; I would guess some changes would be needed, but haven't checked.

@Zaharid
Copy link
Copy Markdown
Contributor

Zaharid commented Feb 27, 2023

Someone should go over the docs as well; I would guess some changes would be needed, but haven't checked.

$ git grep "C++" -- doc/sphinx/

Looks like a good place to start.

@github-actions
Copy link
Copy Markdown

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

Copy link
Copy Markdown
Member

@RoyStegeman RoyStegeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the changes look fine. There are still a few mentions of cpp in the docstrings that can be removed.

Comment thread validphys2/examples/mc_gen_report.md
@scarlehoff scarlehoff removed the run-fit-bot Starts fit bot from a PR. label Mar 1, 2023

def run(self):
if sys.version_info < (3, 6):
if sys.version_info < (3, 9):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we officially drop 3.8 as well?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We are no longer building conda packages for 3.8.

Copy link
Copy Markdown
Contributor

@alecandido alecandido Mar 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I wonder if we should drop also from the other projects...

Point is that I'm fully happy in dropping 3.7, that will go EOL in June (now that I'd like to support 3.11, I don't feel guilty dropping an almost EOL version, even if it's the one running on galileo), but I'm not as sure for 3.8.

In particular, I believe @cschwan for example is still using it (if I remember correctly). Most likely he doesn't care too much about installing vp, but it makes me think that some still very relevant versions of major distributions are still on it.

Copy link
Copy Markdown
Member Author

@scarlehoff scarlehoff Mar 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, some are. I still have 3.8 in a few places. And if you really want to use it you can just install n3fit/validphys manually and it will work. But since we are not supporting it anymore (or testing for it at all) in "the official conda package" we should warn people.

for dataset in group.datasets:
ds = dataset.load()
Ndata = ds.GetNData()
Ndata = dataset.load_commondata().ndata
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it is irrelevant, but is there a reason not to follow the usual naming conventions?
In any case, this is really beyond the scope of this PR...

Copy link
Copy Markdown
Contributor

@alecandido alecandido left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a look at all the diffs, and they look like a careful use of grep for a few specific keywords :D

However, to me, it is perfect. I'm also amazed that you have been able to change code in several places while updating very little tests, so the code changed/dropped was either so well-tested to be implementation independent, or just untested.
In both cases, there is a good reason to be confident in the modifications :)

@scarlehoff
Copy link
Copy Markdown
Member Author

I had a look at all the diffs, and they look like a careful use of grep for a few specific keywords :D

Indeed. This is also an answer your question on why I left some variables with "old convention names". That would've required extra greps/searches (or looking two lines below). Too much effort for my old tired hands.

@Zaharid Zaharid requested a review from comane March 8, 2023 22:16
Copy link
Copy Markdown
Member

@comane comane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me.
None of the modifications done in validphys.core.py affects the closure tests (I have explicitly checked this).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants