Issue when creating a Leading Order table

[scarlehoff]

I've tried to create a Leading Order version of theory 400. In order to do that I've copied all relevant files and just set PTO: 0.
I'm using the packages that one gets doing pip install pineko. I also tried using the latest master of pineko just in case with the same result.

eko: 0.13.5
pineappl: 0.6.1

This works for many fktables but it fails for some of them. I'm going to use as example ATLASZRAP36PB but the same problems can be found with other datasets.

In the lo_failed.zip file you can find the operator and theory card

log

Analyze ATLASZRAP36PB ┌───────────────┐ │ Computing ... │ └───────────────┘ /home/juacrumar/pinekoing/theories_slim/data/grids/390/ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Z0_eta34.pineappl.lz4 + /home/juacrumar/pinekoing/theories_slim/data/ekos/390/ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Z0_eta34.tar = /home/juacrumar/pinekoing/theories_slim/data/fktables/390/ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Z0_eta34.pineappl.lz4 with max_as=1, max_al=0, xir=1.0, xif=1.0 thread '' panicked at 'Nothing returned from evolution.: EvolutionFailure("no operator for x1 = 0.002038117507440189 found")', pineappl_py/src/grid.rs:542:14 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace Traceback (most recent call last): File "/home/juacrumar/pinekoing/pinekovortex/bin/pineko", line 8, in sys.exit(command()) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/pineko/cli/theory_.py", line 95, in fks ).fks(pdf) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/pineko/theory.py", line 453, in fks self.iterate(self.fk, tcard=tcard, pdf=pdf) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/pineko/theory.py", line 212, in iterate f(name, grid, **kwargs) File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/pineko/theory.py", line 422, in fk _grid, _fk, comparison = evolve.evolve_grid( File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/pineko/evolve.py", line 221, in evolve_grid fktable = grid.evolve( File "/home/juacrumar/pinekoing/pinekovortex/lib/python3.10/site-packages/pineappl/grid.py", line 340, in evolve self.raw.evolve( pyo3_runtime.PanicException: Nothing returned from evolution.: EvolutionFailure("no operator for x1 = 0.002038117507440189 found")

If I change PTO: 1 in the theory and rerun the creation of the ekos then it does work. This is in nlo_ok.zip. As you can see the only difference between the two theory cards is the PTO. Since it might be possible that other things need to change for the theory to work at LO, I'm also reproducing the theory card below:

ID: 390
Comments: NNPDF4.0 NNLO alphas=0.118 with new pipeline
CKM: 0.97428 0.22530 0.003470 0.22520 0.97345 0.041000 0.00862 0.04030 0.999152
DAMP: 0
EScaleVar: 1
FNS: FONLL-C
GF: 1.1663787e-05
HQ: POLE
IC: 0
IB: 0
MP: 0.938
MW: 80.398
MZ: 91.1876
MaxNfAs: 5
MaxNfPdf: 5
ModEv: TRN
ModSV: None
NfFF: 4
PTO: 0
Q0: 0.75
QED: 0
Qmb: 4.92
Qmc: 1.51
Qmt: 172.5
Qref: 91.2
Qedref: 91.2
SIN2TW: 0.23126
SxOrd: LL
SxRes: 0
TMC: 1
XIF: 1.0
XIR: 1.0
alphaqed: 0.007778240462411
alphas: 0.118
global_nx: 0
kbThr: 1.0 
kcThr: 1.0
ktThr: 1.0
mb: 4.92
mc: 1.51
mt: 172.5
nfref: 5
nf0: 4
fact_to_ren_scale_ratio: 1.0

nlo_ok.zip

lo_fail.zip
grid_lo.npy.zip
grid_nlo.npy.zip

[scarlehoff]

Note that for the operator cards I'm using the template in here https://github.com/NNPDF/theories_slim/blob/main/data/operator_cards/400/_template.yaml

[cschwan]

Note that for the operator cards I'm using the template in here https://github.com/NNPDF/theories_slim/blob/main/data/operator_cards/400/_template.yaml

Isn't that exactly the problem? The missing x-grid point isn't in the template file.

[felixhekhorn]

some relevant piece of information:

1. grids are different:

felix@FHe19b:~/Physik/N3PDF/FK/lo_fail$ pineappl  subgrids --x1 ATLASWZRAP36PB-ATLAS-arXiv\:1109.5141-Z0_eta34.pineappl.lz4 
o b l                                                                                                                                                      x1
-+-+--+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 0 0  4.481e-2, 4.181e-2, 3.897e-2, 3.629e-2, 3.377e-2, 3.141e-2, 2.918e-2, 2.710e-2, 2.515e-2, 2.332e-2, 2.162e-2, 2.002e-2, 1.854e-2, 1.715e-2, 1.587e-2, 1.467e-2, 1.355e-2, 1.252e-2, 1.156e-2, 1.067e-2, 9.848e-3, 9.084e-3, 8.378e-3, 7.724e-3, 7.120e-3, 6.561e-3, 6.045e-3, 5.568e-3, 5.128e-3, 4.722e-3
[...]
1 7 11 1.000e0, 9.596e-1, 9.196e-1, 8.798e-1, 8.404e-1, 8.013e-1, 7.626e-1, 7.244e-1, 6.865e-1, 6.491e-1, 6.122e-1, 5.759e-1, 5.401e-1, 5.050e-1, 4.705e-1, 4.368e-1, 4.039e-1, 3.718e-1, 3.407e-1, 3.106e-1, 2.816e-1, 2.538e-1, 2.273e-1, 2.022e-1, 1.785e-1, 1.564e-1, 1.359e-1, 1.171e-1, 9.999e-2, 8.467e-2

vs.

felix@FHe19b:~/Physik/N3PDF/FK/nlo_ok$ pineappl subgrids --x1 ATLASWZRAP36PB-ATLAS-arXiv\:1109.5141-Z0_eta34.pineappl.lz4 
o b l                                                                                                                                                                                                       x1
-+-+--+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 0 0  3.621e-4, 5.488e-4, 8.314e-4, 1.259e-3, 1.903e-3, 2.874e-3, 4.329e-3, 6.496e-3, 9.699e-3, 1.438e-2, 2.109e-2, 3.052e-2, 4.341e-2, 6.048e-2, 8.228e-2, 1.091e-1, 1.411e-1, 1.780e-1, 2.195e-1, 2.651e-1, 3.144e-1, 3.669e-1, 4.222e-1, 4.799e-1, 5.398e-1, 6.015e-1, 6.648e-1, 7.296e-1, 7.956e-1, 8.628e-1, 9.309e-1, 1.000e0
[...]
0 7 31 1.292e-5, 1.960e-5, 2.974e-5, 4.511e-5, 6.844e-5, 1.038e-4, 1.575e-4, 2.388e-4, 3.621e-4, 5.488e-4, 8.314e-4, 1.259e-3, 1.903e-3, 2.874e-3, 4.329e-3, 6.496e-3, 9.699e-3, 1.438e-2, 2.109e-2, 3.052e-2, 4.341e-2, 6.048e-2, 8.228e-2, 1.091e-1, 1.411e-1, 1.780e-1, 2.195e-1, 2.651e-1, 3.144e-1, 3.669e-1, 4.222e-1, 4.799e-1, 5.398e-1, 6.015e-1, 6.648e-1, 7.296e-1, 7.956e-1, 8.628e-1, 9.309e-1, 1.000e0

@scarlehoff why does LO have two orders?
2) the metadata.yaml in either eko is identical

so there seems to be a problem on reinterpolation

Isn't that exactly the problem? The missing x-grid point isn't in the template file.

they are added later on (since they are always dynamic)

[alecandido]

But are you using exactly the template as operator, or do you replace the x-grid? (as it should happen automatically)

[scarlehoff]

pineko/src/pineko/evolve.py

Line 195 in 59b76b2

order_mask = pineappl.grid.Order.create_mask(grid.orders(), max_as, max_al, True)

Asking pineappl for the information at one order higher for the evol_info seems to do the trick for now... still running though.

max_as += 1
order_mask = pineappl.grid.Order.create_mask(grid.orders(), max_as, max_al, True)
evol_info = grid.evolve_info(order_mask)
max_as -= 1
order_mask = pineappl.grid.Order.create_mask(grid.orders(), max_as, max_al, True)

[cschwan]

That's very strange and it would mean that there's a bug somewhere.

[cschwan]

@scarlehoff why does LO have two orders? 2) the metadata.yaml in either eko is identical

Even stranger (?) is that NLO has one order. It looks like they have been swapped.

[scarlehoff]

Sorry, this is my mistake. In the NLO folder there is the output fktable, not the grid.
I'm using the same grid in both cases.

Edit: I've updated OP with the right .zip file for NLO separating grid and fktable.

[scarlehoff]

Some more details.

Failure (running at LO):

grid.orders()=[<pineappl.grid.Order object at 0x7fd413bf5f00>, <pineappl.grid.Order object at 0x7fd413bf5960>]
max_as=1
max_al=0
order_mask=array([ True, False])

Success (manually doing max_as=2)

grid.orders()=[<pineappl.grid.Order object at 0x7fa81025df00>, <pineappl.grid.Order object at 0x7fa81025d210>]
max_as=2
max_al=0
order_mask=array([ True,  True])

I've tried to pad the order_mask with 9 extra False but nothing changes.

[cschwan]

This is indeed a problem in PineAPPL, and I've fixed it in this commit: NNPDF/pineappl@ff235ac. The solution for this bug then is to depend on PineAPPL >=0.6.2, which I'm preparing right now.

[cschwan]

Here's the new release: https://pypi.org/project/pineappl/0.6.2/.

[scarlehoff]

I just tried it (did pip install -U pineappl and tried to rerun the one that was failing) and seems to work.
Thanks!