Improving reading and recreating fit pseudodata

[siranipour]

Now that python is used to construct the pseudodata, I've updated the pseudodata.py function that regenerates the pseudodata of a given fit. It's now much easier to do this in parallel because we don't use swig objects anymore.

Closes #1323

The tests will, however, fail for now until I regenerate a PDF that saves its python pseudodata for the regression test. @wilsonmr, I may be misremembering but did you at some point have a PR which saved the pseudodata of a fit?

EDIT: This has turned into a bit of a general improvements PR.

1. Corrected the recreation of a fit pseudodata. I've swapped the C++ make_replica with the python one.
2. Improved the reading of pseudodata from a fit which has fitting::savepseudodata: True
3. There was a function that created the tr_mask for all replicas:
   

   nnpdf/validphys2/src/validphys/n3fit_data.py

   Line 386 in b636ae0

   def training_mask_table(replicas_exps_tr_masks, replicas, experiments_index):

   
   I've now changed this so it works on an individual replica and then collected it over replicas (I did this because it was useful for the recreation of a training validation mask.
4. Fixed the tests

[wilsonmr]

now that the replica generation is fast, can we not just call the action on a set of replicas? Is there really that much speed up from MP now? In fact since the action is in validphys, can we not just call --parallel in the usual way to get the MP?

[wilsonmr]

@wilsonmr, I may be misremembering but did you at some point have a PR which saved the pseudodata of a fit?

This got added in #1081 I think with the savepseudodata flag:

nnpdf/n3fit/src/n3fit/scripts/n3fit_exec.py

Line 131 in 74d9183

if file_content["fitting"].get("savepseudodata"):

[wilsonmr]

in particular, calling the API inside of a validphys action seems really horrible, I don't think this should be an internal action, or even exist at all

[siranipour]

Oh true actually, let me see if the --parallel flag works

[wilsonmr]

just to say, the function which I mentioned, pseudodata_table, was actually not updated to use make_replica in #1268 which is why the function is so painfully slow, it also means that function is currently useless because it's not even dumping the right pseudodata during the fit haha

[wilsonmr]

how long does this take now with the collect approach @siranipour ?

[siranipour]

just to say, the function which I mentioned, pseudodata_table, was actually not updated to use make_replica in #1268 which is why the function is so painfully slow, it also means that function is currently useless because it's not even dumping the right pseudodata during the fit haha

Ah ok haha, I'll try and get another PR for that then because the tests will fail until we regenerate a new regression fit.

how long does this take now with the collect approach @siranipour ?

Unfortunately I was a bit pushed for time, so I didn't get to fully benchmark it, but it still took a non negligible O(few minutes). I'll take a proper look and also play with the parallel flag and see what happens.

[github-actions]

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

• Fit Name: NNBOT-aab94f0a6-2021-07-21
• Fit Report: https://vp.nnpdf.science/ENmhRHSUSfqmR6gt4zmOOw==
• Fit Data: https://data.nnpdf.science/fits/NNBOT-aab94f0a6-2021-07-21.tar.gz

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

[siranipour]

Any ideas why the regression tests fail? They pass locally for me...

[siranipour]

Ahh I generated the new baseline with theory 200 which is why the tests are failing. Will fix

[siranipour]

@scarlehoff this PR flies in tandem with #1333, in that it reproduces the pseudodata a posteriori which is in principle the ones that are saved by #1333. I would appreciate if you could take a look at this soon too.

The basic idea is that I've added actions that work on a per replica basis. I.e replicate the pseudodata of replica_i. We then collect over either NSList(range(nrep), nskey='replica') to replicate all replicas, or produce_pdfreplicas, which is basically the same thing but accounting for the postfit reshuffling