Reimplement Atlas Z0 8 TeV HIMASS

[comane]

(this branch) https://vp.nnpdf.science/_LpEXpBCT86YesZeTaCKSQ==
(master) https://vp.nnpdf.science/zmrjDp_ySRqdDZTmW8MQDA==

The new covariance matrix does not coincide with the old one, however they are extremely close to each other.
The suspected reason for this is numerical precision:

1. There are small, possibly numerical, differences between the central values. Since all the uncertainties are MULT this affects the uncertainties.

2. In the old implementation stat was treated as ADD, however it is MULT (this does not affect the systematics covmat)

from validphys.api import API
import numpy as np

name_ds = "ATLAS_Z0_8TEV_HIMASS_M-Y"
inp1 = {"dataset_input": {"dataset": f"{name_ds}"}, "theoryid": 40_000_000, "use_cuts": "internal", "t0pdfset": "NNPDF40_nnlo_as_01180", "use_t0": True}

inp2 = {"dataset_input": {"dataset": f"{name_ds}", "variant": "legacy"}, "theoryid": 40_000_000, "use_cuts": "internal", "t0pdfset": "NNPDF40_nnlo_as_01180", "use_t0": True}

covmat2 = API.covmat_from_systematics(**inp2)

covmat1 = API.covmat_from_systematics(**inp1)

relative_difference = np.abs(covmat1 - covmat2) / np.maximum(np.abs(covmat1), np.abs(covmat2))

print(np.max(relative_difference))
0.0835

CHi2 comparison

API.central_chi2(**{**inp1, "pdf": "NNPDF40_nnlo_as_01180"})
53.937

API.central_chi2(**{**inp2, "pdf": "NNPDF40_nnlo_as_01180"})
53.836

[scarlehoff]

I'm confused about the keyword SPECIAL here.
We have the CORR and UNCORR as variables that are correlated within a dataset, and then type: <some name>, variables that should be correlated between dataset within an experiment*.

For instance, in this case ATLASLUMI12. If you put SPECIAL it will be correlated with other stuff called SPECIAL. But you want to use SPECIAL also for the 13 TeV ones which would be wrong? (I think ATLASLUMI12 is 8 TeV looking at where it appears.

Is there something I'm missing?

*and we cannot correlate between experiments here because the assumption of an experiment-diagonal covmat is hardcoded in a bunch of places. Luckily there's now a PR sort of dealing with that.

[comane]

*and we cannot correlate between experiments here because the assumption of an experiment-diagonal covmat is hardcoded in a bunch of places. Luckily there's now a PR sort of dealing with that.

Just out of curiosity, which PR is dealing / dealt with that?

[comane]

It looks like tests are failing because stat uncertainties are always supposed to be ADD.
For this datasets, however, this is not the case.

Since having ADD or MULT makes no difference we might just want to put it back to ADD

[t7phy]

hey @comane! I will review the 2 PRs in the next few days

[t7phy]

@comane given that the test fails because of the type being MULT for stat, I would personally just rename the stat uncertainty to something else, like stat_mult, and add a comment in the filter explaining the reason that the code expects stat to be ADD but this is an exception, etc. etc. instead of putting it as ADD even when it should be MULT. this would ensure the data implementation is consistent, covmat would remain the same and the tests should pass I assume.
this is just a suggestion, feel free to take your call here, other than that, everything seems fine and this can be merged.

[jacoterh]

Hi @comane, I landed here after I wondered how the experimental covmat gets constructed in validphys. When the experimentalists don't provide a covmat, we use the breakdown of systematics to construct it as follows

In general, the off diagonal components can have either sign in a covariance matrix, and so we should have that s_{i, corr} can either be positive or negative. But when we go on hepdata, typically all values we use for s_{i, corr} are actually positive! The experimentalists only give the size of s_{i, corr}, so any information on the sign is lost. Are we sure in that case the covmat can be constructed this way?

In this line selected here I noticed that the error you extract from hepdata is negative, is this done on purpose?