Status of the new commondata format implementation

[scarlehoff]

This issue collects the status of the new data implementation.

The fully implemented datasets can be found in PR #1813
Th reader for the new commondata format is in PR #1678

All datasets merged to #1813 have been tested and work with the reader of PR #1678.

The documentation for the new CD format is instead in PR #1708

---

For people implementing new datasets, whenever a new datasets is implemented and it is considered final,
please open a PR pointing to #1813. See for instance #1821
It is not necessary to check that it works with the reader, although it would be much appreciated, of course.

It is understood that datasets for which there is no PR pointing to #1813 are still not completed.
All datasets which have an "older" version will be compared and checked and if it is deemed to be equal (or compatible up to bug fixes) it will be added to this list, so only those are considered to be fully implemented.

---

Reports and examples

Implemented branches

New jet data (implemented in #1821)
https://vp.nnpdf.science/3vtMtF9yQFe1V-kmiGbh2w==

Branchs under review

DIS+j

New DIS + j #1825

DY (reimplementation) #1846

LHCB DY (1) #1826
ATLAS DY (1) #1866
CMS DY (1) #1869

Jets

Atlas/CMS jet/dijet #1785

Top

ttb ATLAS #1837
ttb CMS #1836
ttb total #1834

Datasets from NNPDF4.0 (all done in #1813)

• NMCPD_dw_ite
• NMC
• SLACP_dwsh
• SLACD_dw_ite
• BCDMSP_dwsh
• BCDMSD_dw_ite
• CHORUSNUPb_dw_ite
• CHORUSNBPb_dw_ite
• NTVNUDMNFe_dw_ite
• NTVNBDMNFe_dw_ite
• HERACOMBNCEM
• HERACOMBNCEP460
• HERACOMBNCEP575
• HERACOMBNCEP820
• HERACOMBNCEP920
• HERACOMBCCEM
• HERACOMBCCEP
• HERACOMB_SIGMARED_C
• HERACOMB_SIGMARED_B
• DYE886R_dw_ite
• DYE886P
• DYE605_dw_ite
• DYE906R_dw_ite
• CDFZRAP_NEW
• D0ZRAP_40
• D0WMASY
• ATLASWZRAP36PB
• ATLASZHIGHMASS49FB
• ATLASLOMASSDY11EXT
• ATLASWZRAP11CC
• ATLASWZRAP11CF
• ATLASDY2D8TEV
• ATLAS_DY_2D_8TEV_LOWMASS
• ATLAS_WZ_TOT_13TEV
• ATLAS_WP_JET_8TEV_PT
• ATLAS_WM_JET_8TEV_PT
• ATLASZPT8TEVMDIST
• ATLASZPT8TEVYDIST
• ATLASTTBARTOT7TEV
• ATLASTTBARTOT8TEV
• ATLAS_TTBARTOT_13TEV_FULLLUMI
• ATLAS_TTB_DIFF_8TEV_LJ_TRAPNORM
• ATLAS_TTB_DIFF_8TEV_LJ_TTRAPNORM
• ATLAS_TOPDIFF_DILEPT_8TEV_TTRAPNORM
• ATLAS_1JET_8TEV_R06_DEC
• ATLAS_2JET_7TEV_R06
• ATLASPHT15_SF
• ATLAS_SINGLETOP_TCH_R_7TEV
• ATLAS_SINGLETOP_TCH_R_13TEV
• ATLAS_SINGLETOP_TCH_DIFF_7TEV_T_RAP_NORM
• ATLAS_SINGLETOP_TCH_DIFF_7TEV_TBAR_RAP_NORM
• ATLAS_SINGLETOP_TCH_DIFF_8TEV_T_RAP_NORM
• ATLAS_SINGLETOP_TCH_DIFF_8TEV_TBAR_RAP_NORM
• CMSWEASY840PB
• CMSWMASY47FB
• CMSDY2D11
• CMSWMU8TEV
• CMSZDIFF12
• CMS_2JET_7TEV
• CMS_1JET_8TEV
• CMSTTBARTOT7TEV
• CMSTTBARTOT8TEV
• CMSTTBARTOT13TEV
• CMSTOPDIFF8TEVTTRAPNORM
• CMSTTBARTOT5TEV
• CMS_TTBAR_2D_DIFF_MTT_TRAP_NORM
• CMS_TTB_DIFF_13TEV_2016_2L_TRAP
• CMS_TTB_DIFF_13TEV_2016_LJ_TRAP
• CMS_SINGLETOP_TCH_TOT_7TEV
• CMS_SINGLETOP_TCH_R_8TEV
• CMS_SINGLETOP_TCH_R_13TEV
• LHCBZ940PB
• LHCBZEE2FB_40
• LHCBWZMU7TEV
• LHCBWZMU8TEV
• LHCB_Z_13TEV_DIMUON
• LHCB_Z_13TEV_DIELECTRON

[scarlehoff]

@t7phy could you add the specific datasets (instead of the name of the whole set). That way we can achieve more granularity on the checks.

[t7phy]

@scarlehoff I had somehow missed your above comment and just saw it. Actually, at this point, every dataset in the branch gluon_pdf_ncd is correctly implemented. I think it would be simpler if you just randomly test few random distributions from few random datasets. Otherwise there would be tonnes of tests to run. On the other hand, if you would prefer to test all the datasets, then perhaps I could list them above. Let me know.

[scarlehoff]

Hi @t7phy I would like to test all of the datasets since we want to ensure full backwards compatibility. I would like to register even the factors of -1 in the eigenvalues since any difference can potentially create issues downstream.

[scarlehoff]

Hi @t7phy what does NA mean? Are these completely new datasets?

Note that in order to test I also need to have the old dataset name I should test against.

[t7phy]

Hi @t7phy what does NA mean? Are these completely new datasets?

Note that in order to test I also need to have the old dataset name I should test against.

Hi @scarlehoff , the ones with NA are all new. It would still be good to test them as they were implemented while we were making many changes to the keys in metadata and this will ensure that I implemented everything correctly. The empty ones are the ones which are also there in the old format. I will be adding the names to them, just need to track them all.

[scarlehoff]

Ok. When I test I'll separate both scenarios since for the ones we have old-new I'll also include the differences with respect to the new ones in the test. Please leave a comment in this issue when you are done adding the names to the tables (github doesn't notify the edits)

[t7phy]

Ok. When I test I'll separate both scenarios since for the ones we have old-new I'll also include the differences with respect to the new ones in the test. Please leave a comment in this issue when you are done adding the names to the tables (github doesn't notify the edits)

Yes yes, I will do that. I hadn't commented precisely because I still had to add the old dataset names.

[t7phy]

@scarlehoff, the old dataset names are now added

[scarlehoff]

I'm noticing that some of the datasets have as hepdata "NA". Is this because they don't come from hepdata? (I'm guessing so)

Should we put there instead the url from which the data was taken? Or some explanation saying where the original data is coming from?

One example of this is

ATLAS_TTBAR_13TEV_TOT_X-SEC

---

Some specific errors I'm finding for some datasets:

• ATLAS_TTBAR_13TEV_TOT_X-SEC

There's also a typo in the kinematics of ATLAS_TTBAR_13TEV_TOT_X-SEC. It is only one bin with 3 variables, instead they appear as 3 different bins each with one variable.

It has:

bins:
- zero:
    min: null
    mid: 0
    max: null
- mt_sqr:
    min: null
    mid: 29846.0176
    max: null
- sqrt_s:
    min: null
    mid: 13000
    max: null

instead of

bins:
- zero:
    min: null
    mid: 0
    max: null
  mt_sqr:
    min: null
    mid: 29846.0176
    max: null
  sqrt_s:
    min: null
    mid: 13000
    max: null

Then, one thing that worries me is that the one point for this dataset in the old implementation had the numerical value 830 whereas this one has 818.

The same problems can be found for ATLAS_TTBAR_7TEV_TOT_X-SEC although in this case the total cross section has the exact same numerical value as in the old implementation.

---

In general I was considering the mid value of the kinematics to be the only "true" value (and thus the only that needs to be filled). I'm saying some datasets which do not include mid (mid : null). Is it fine to take the middle point between max and min as the mid value ?

(more comments to come)

[t7phy]

I'm noticing that some of the datasets have as hepdata "NA". Is this because they don't come from hepdata? (I'm guessing so)

Should we put there instead the url from which the data was taken? Or some explanation saying where the original data is coming from?

One example of this is

ATLAS_TTBAR_13TEV_TOT_X-SEC

Some specific errors I'm finding for some datasets:

* `ATLAS_TTBAR_13TEV_TOT_X-SEC`

There's also a typo in the kinematics of ATLAS_TTBAR_13TEV_TOT_X-SEC. It is only one bin with 3 variables, instead they appear as 3 different bins each with one variable.

It has:

bins:
- zero:
    min: null
    mid: 0
    max: null
- mt_sqr:
    min: null
    mid: 29846.0176
    max: null
- sqrt_s:
    min: null
    mid: 13000
    max: null

instead of

bins:
- zero:
    min: null
    mid: 0
    max: null
  mt_sqr:
    min: null
    mid: 29846.0176
    max: null
  sqrt_s:
    min: null
    mid: 13000
    max: null

Then, one thing that worries me is that the one point for this dataset in the old implementation had the numerical value 830 whereas this one has 818.

The same problems can be found for ATLAS_TTBAR_7TEV_TOT_X-SEC although in this case the total cross section has the exact same numerical value as in the old implementation.

In general I was considering the mid value of the kinematics to be the only "true" value (and thus the only that needs to be filled). I'm saying some datasets which do not include mid (mid : null). Is it fine to take the middle point between max and min as the mid value ?

(more comments to come)

For all the total cross sections, the values are taken directly from the abstract of the papers, thus there is no hepdata entry.

For the kinematic bin typo, since these entries for total cross sections were typed by handed, I must have made a mistake, I will fix them.

Concerning the mid value, from what I have seen, some experiments seem to provide the bin edges and the central value, however the central value isn't necessarily the middle value between min and max, so I think it depends on the distribution. However, since some experiments don't provide the central values, I guess in this cases the reader could take the min and max, and average them to get the central 'true' value.

[scarlehoff]

For all the total cross sections, the values are taken directly from the abstract of the papers, thus there is no hepdata entry.

Then we should add this information as a comment somewhere.

Concerning the mid value ...

Ok! Then I'll take the average for these. Thanks!

---

Now for some more specific questions (pinging as well @enocera here as his input might be needed), expanding on what I wrote above.

• ATLAS_TTBAR_13TEV_TOT_X-SEC - ATLASTTBARTOT13TEV

Total cross section and statistical errors are very different:

old: 818 +- 8
new: 830 +- 0.4

by looking at the paper clearly the second value is correct, does that mean that the old value was wrong?
The numer of systematic uncertainties is also different (in the old version we have 3, in the new version we have only 2). Are these kind of changes expected? If this is an expected correction, maybe we should keep some record of the fact that the value have changed during the implementation of the new datasets?

• ATLAS_TTBAR_7TEV_TOT_X-SEC - ATLASTTBARTOT7TEV

For this dataset the old and new numerical values for both the data and the uncertainties is the same, what has changed however is the name (and type) of the uncertainties.

In the old dataset all uncertainties were considered multplicatives and the names were: UNCORR | ATLASLUMI11 | UNCORR

instead, in the new one, two are multiplicative with names: CORR | ATLASLUMI7 and one additive with name CORR.

My question here is, should I worry about the change of name? I want to test as many things as possible but I don't want to raise warnings / errors for things that are just a convention which might have changed.

[scarlehoff]

Something has also gone wrong for these datasets,

"ATLAS_TTBAR_8TEV_LJ_DIF_PTT-NORM"
"ATLAS_TTBAR_8TEV_LJ_DIF_YT"
"ATLAS_TTBAR_8TEV_LJ_DIF_YT-NORM"
"ATLAS_TTBAR_8TEV_LJ_DIF_YTTBAR"
"ATLAS_TTBAR_8TEV_LJ_DIF_YTTBAR-NORM"

as the names of some of the uncertainties contain commas, slashes and spaces which I'm not sure are intentional (it looks more like a parsing problem). If it is intentional (and the names should contain commas and slashes) please remove trailing whitespaces.

[enocera]

Something has also gone wrong for these datasets,

"ATLAS_TTBAR_8TEV_LJ_DIF_PTT-NORM" "ATLAS_TTBAR_8TEV_LJ_DIF_YT" "ATLAS_TTBAR_8TEV_LJ_DIF_YT-NORM" "ATLAS_TTBAR_8TEV_LJ_DIF_YTTBAR" "ATLAS_TTBAR_8TEV_LJ_DIF_YTTBAR-NORM"

as the names of some of the uncertainties contain commas, slashes and spaces which I'm not sure are intentional (it looks more like a parsing problem). If it is intentional (and the names should contain commas and slashes) please remove trailing whitespaces.

I guess that @t7phy is using the names given by experimentalists to systematic uncertainties (as given on Hepdata) which do include commas and slashes. The advantage of this is to have a one-to-one correspondence with Hepdata. Would that be a problem to keep these names?