MakeReplica in Python

[Zaharid]

We should be able to generate a set of pseudodata replicas, with the algorithm we use currently. This requires to have a commondata #378.

[Zaharid]

@scarlehoff @scarrazza This now seems a main blocker for getting rid of libnnpdf (and it is makig a lot of logic in n3fit and also eg #650 needlessly cumbersome). Not to mention it may well be a non negligible amount of the total n3fit runtime. Maybe it can get some love from (anyone in) Milan?

A few things we would like here is to get rid of the absurd ~O(n^3) string comparison for the systematics

nnpdf/libnnpdf/src/chisquared.cc

Line 70 in d16236e

if ((isys.name == "THEORYCORR" || isys.name == "THEORYUNCORR") && !use_theory_errors)

which turns out to be most of the runtime, and to make it easy and quick to generate many pseudodata replicas at the same time.

[scarlehoff]

In a DIS fit loading the data + replica generation is about 10% of the time. Not negligible but not terrible either.

Anyway, I guess it would be good if n3fit didn't have to care about generating the replicas but rather the datasets coming in were already "replic'd".

[Zaharid]

Another important, but easy requirement is that this should be able to genererate many pseudorreplicas at once, paying one time for expensive bits like creating the covmat. That would take care of our problems pretty well.

[RosalynLP]

Hello I would like to do some work on this and am wondering whether anyone could give me some guidance on where to start?

[RosalynLP]

Am I right in thinking we would need to start with #378?

[Zaharid]

Indeed having had some progress with the parsers would certainly have had helped in getting this started.

[RosalynLP]

So should I work on the parser then? If so is anyone else working on these and could someone advise what to being with?

[scarrazza]

You can grab some code from the closed/not merged PR, e.g. function https://github.com/NNPDF/nnpdf/pull/476/files#diff-6c261ce87f17107be2eda452bc1aec61R251