Add an example using PineAPPL to compute predictions

[scarlehoff]

Added a notebook to the examples folder where validphys is monkeypatched to utilize pineappl grids instead of fktables.

The code for this will not be integrated in validphys (validphys will keep using only fktables because changing to grids can open many cans of worms that I don't want to open) but as you can see in the notebook the code is quite simple.

This deals (in a way) with parts of #1961

ps: If there is any function in validphys for which the monkey patching in the notebook doesn't produce the desired result it might be due to said function using directly convolute instead of results. Please open an issue / PR because the more we make the validphys API be consistent the easiest it becomes to do things like this in a simple manner!

[achiefa]

Hi @scarlehoff , I've found two problems with the nb:

1. In _get_pine_container, the following if statement crashes if the metadata does not contain the normalisation key

def _get_pine_container(dataset):
    cd = dataset.commondata
    metadata = cd.metadata
    theory_meta = metadata.theory

    pinegrids = []
    for operator in theory_meta.FK_tables:
        tmp = []
        for i in operator:
            factor = theory_meta.conversion_factor
            if theory_meta.normalization is not None:  # <--------- Here
                factor *= theory_meta.normalization.get(i, 1.0)

I ''solved'' this problem by simply commenting out the two last lines.

2. The second problem I get comes about when I try to compute

full_chi2 = API.abs_chi2_data_thcovmat(**kwargs, **theory_opt, use_pdferr=True) 

and I get this error

PanicException: called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }

Note that the same does not happen with base_chi2 and th_chi2.

[scarlehoff]

Hi @achiefa the first one is strange, because the normalization key should either be None or have some value,

nnpdf/validphys2/src/validphys/commondataparser.py

Line 241 in 7bb760d

normalization: Optional[dict] = None

What error do you see exactly?

The second one instead really looks like pineappl is not able to find the grid? Maybe I hardcoded the folder somewhere without realising? (because I called the folder the same as the variable :__ )

[achiefa]

I see this error for the normalisation

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[42], [line 8](vscode-notebook-cell:?execution_count=42&line=8)
      [5](vscode-notebook-cell:?execution_count=42&line=5) kwargs = {"dataset_input": {"dataset": dname}, "theoryid" : tid, "use_cuts":"internal", "pdf": pdf_name}
      [6](vscode-notebook-cell:?execution_count=42&line=6) theory_opt = {"point_prescription": "3 point", "theoryids": {"from_": "scale_variation_theories"}, "use_theorycovmat": True}
----> [8](vscode-notebook-cell:?execution_count=42&line=8) base_chi2 = API.abs_chi2_data(**kwargs)
      [9](vscode-notebook-cell:?execution_count=42&line=9) print(f"Exp chi2: {base_chi2.central_result [/](https://file+.vscode-resource.vscode-cdn.net/) base_chi2.ndata:.4}")
     [11](vscode-notebook-cell:?execution_count=42&line=11) th_chi2 = API.abs_chi2_data_thcovmat(**kwargs, **theory_opt)

File [/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:44](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:44), in API.__getattr__.<locals>.closure(**kwargs)
     [43](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:43) def closure(**kwargs):
---> [44](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:44)     return self.__call__(name, **kwargs)

File [/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:38](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:38), in API.__call__(self, actions, **kwargs)
     [36](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:36) builder.rootns.update(self.loadedenv.ns_dump())
     [37](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:37) builder.resolve_fuzzytargets()
---> [38](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:38) builder.execute_sequential()
     [39](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:39) res = builder.rootns[actions]
     [40](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/api.py:40) return res

File [/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/resourcebuilder.py:166](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/resourcebuilder.py:166), in ResourceExecutor.execute_sequential(self)
    [164](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/resourcebuilder.py:164)     result = callspec.function(self.rootns, callspec.nsspec)
    [165](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/resourcebuilder.py:165) else:
--> [166](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/resourcebuilder.py:166)     result = self.get_result(callspec.function,
    [167](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.11/site-packages/reportengine/resourcebuilder.py:167)                              *self.resolve_callargs(callspec),
...
---> [17](vscode-notebook-cell:?execution_count=41&line=17)     if theory_meta.normalization is not None:
     [18](vscode-notebook-cell:?execution_count=41&line=18)         factor *= theory_meta.normalization.get(i, 1.0)
     [20](vscode-notebook-cell:?execution_count=41&line=20)     pine_path = grid_path [/](https://file+.vscode-resource.vscode-cdn.net/) f"{i}.{EXT}"

AttributeError: 'TheoryMeta' object has no attribute 'normalization'

[scarlehoff]

edit: probably just a mismatch between the repo and the version of the code installed, should be fixable with pip install -e .

[scarlehoff]

To use the pdferr key in the notebook, you need the changes from #2051 (it is only useful to compute the chi2 with the pdf errors included).

[scarlehoff]

Since this has already been used by a few people and seems to work I'll merge it to master.