LO fits with 4.0 methodology

[wilsonmr]

Starting approximately here: #675 (comment)

there is a discussion of LO fits which initially have very poor chi2, it appears that the training chi2 and validation chi2 are behaving very strangely but as pointed out it's probably a weird interaction between a problematic datapoint either appearing in training or validation and then postfit filtering the latter.

The runcards (present on that PR) have been updated and any further discussion on issues when fitting at LO should be made here

cc: @RoyStegeman @scarlehoff

ps: did I miss something vital?

[scarlehoff]

That's a good summary. Thanks @wilsonmr

I'll answer also here @juanrojochacon comment:

agreed, let's see what happens now. If the problem persist, we should store the chi2 logs dataset per dataset to pin down the problem @scarlehoff

The dataset-per-dataset chi2 during the fit would be costly in comparison with the cost of just rerunning the fit for the "bad" experiment when it happens but having an option to turn on the "dataset grouping" on request would be ok.

[juanrojochacon]

yes this will not be needed often but on occasions it can be handy

[RoyStegeman]

The troublesome dataset seems to have been removed, and upon doing iterations the wiggles in the gluon have decreased.

This fit is after two iteration: https://vp.nnpdf.science/W-NiK6lRSTquvrk5-Cg7Xw==
with posf2c active: https://vp.nnpdf.science/pp4sa38_TmSKGaaB-HY9rw==/

@enocera as you can see the charm has become quite negative, which is why again asked about posf2c.

LO pch non-iterated: (somewhat odd chi2 stats) https://vp.nnpdf.science/Q2-fXUFwS4azq723pQ90Yw==
LO pch trice iterated: https://vp.nnpdf.science/rbv5vfd8TXadK6EmCIlTNQ==

[Zaharid]

I find more worrying the related fact that the gluon is very negative... https://vp.nnpdf.science/W-NiK6lRSTquvrk5-Cg7Xw==/figures/Scales0_pdf_report_PDFnormalize0_Basespecs0_PDFscalespecs0_plot_pdfs_g.pdf

Incidentally about the only value in leading order pdfs is having a gluon that grows with decreasing x...

[RoyStegeman]

At all energies?

For example, what you describe is the case at 2.5 GeV: https://vp.nnpdf.science/aZHbFqNLTsejFXynKxLzMg==

[Zaharid]

I would say particularly at low energies which is what they want for monte carlos. That said a possible hack would be to take the PDF at higher energies, like indeed 2.5 GeV and freeze it below.

[juanrojochacon]

Hi @Zaharid @RoyStegeman looking at these results I'd say:

• Indeed the chi2s look much better (for a LO fit of course!) and the huge chi2 values in the tr/val scatter plot have disappeared, which is nice
• The gluon is smoother at large-x, which is also a nice feature
• The fact that gluon is negative at the input parametrisation scale is not surprising, since we impose PDF positivity at somewhat higher scales. An easy solution is to decrease the scale of the positivity observables down to 1.65 GeV
• Charm is very negative: I think we should simply impose the positivity of the charm PDf exactly in the same way as for all light quarks, and this will cure the problem. And this is cleaner theoretically, since at LO all PDFs are just positive-definite

So my two cents to improve the LO fits is to add charm PDF positivity and to decrease the scale for which the positivity of all PDFs is imposed.

[RoyStegeman]

The fact that gluon is negative at the input parametrisation scale is not surprising, since we impose PDF positivity at somewhat higher scales. An easy solution is to decrease the scale of the positivity observables down to 1.65 GeV

Okay, that can be done. However, it requires adjusting the posf2c (or even all observables as you say) commondata file for only one fit. I'm not sure how the positivity scale was chosen (it's now Q^2=5), but should we make this scale the same for all fits or are you happy with only changing it for a single fit?

[enocera]

Not only the commondata files have to be adjusted, but also the FK tables used to practically implement the positivity.

[enocera]

• Charm is very negative: I think we should simply impose the positivity of the charm PDf exactly in the same way as for all light quarks, and this will cure the problem. And this is cleaner theoretically, since at LO all PDFs are just positive-definite

This is feasible, but it will require some work (positivity of the charm PDF is currently not implemented). Before embarking in such a work, I'd be more comfortable to discuss the issue with the rest of the collaboration.

[juanrojochacon]

fine by me to discuss, though I'd say that at LO pdfs are positive definite by construction so any LO PDF set with negative PDFs is wrong

[juanrojochacon]

The fact that gluon is negative at the input parametrisation scale is not surprising, since we impose PDF positivity at somewhat higher scales. An easy solution is to decrease the scale of the positivity observables down to 1.65 GeV

Okay, that can be done. However, it requires adjusting the posf2c (or even all observables as you say) commondata file for only one fit. I'm not sure how the positivity scale was chosen (it's now Q^2=5), but should we make this scale the same for all fits or are you happy with only changing it for a single fit?

This is only for the LO fits, which are conceptually different from NLO fits and so on. LO PDfs need to be positive at all scales, so there the positivity constraint needs to be more stringent

[RoyStegeman]

This is feasible, but it will require some work (positivity of the charm PDF is currently not implemented). Before embarking in such a work, I'd be more comfortable to discuss the issue with the rest of the collaboration.

Can't we just use posf2c for that? See https://vp.nnpdf.science/pp4sa38_TmSKGaaB-HY9rw==

Not only the commondata files have to be adjusted, but also the FK tables used to practically implement the positivity.

You're right, of course. My question was mainly because I don't know if the Q^2 as it is used in positivity observables now, has been carefully chosen or if it's probably fine to lower the scale. Because I guess ideally we want to have the same scale for all fits, so depending on the reason Q^2 was chosen to be Q^2=5 we might already know whether it's feasible to change this scale for all fits.

This is only for the LO fits, which are conceptually different from NLO fits and so on. LO PDfs need to be positive at all scales, so there the positivity constraint needs to be more stringent

In this case I think this would require storing those LO positivity observables as separate files and generating the fk tables for the LO theories using those commondata files. I could do this locally just to produce the fits once, but that would not allow other users to reproduce those results once the code is public.

[enocera]

This is only for the LO fits, which are conceptually different from NLO fits and so on. LO PDfs need to be positive at all scales, so there the positivity constraint needs to be more stringent

I know, but then why not requiring positive neural networks tout court in the LO fit?

[scarlehoff]

I am a bit worried about some specific datasets where the chi2 is very high and the prediction is clearly off (for instance see here

Does it make sense to consider those points at all when the LO prediction is so clearly off, even if it is non 0?

wrt positivity, I'm not entirely sure what the use case is (I would argue using them is a mistake at best) but given the chi2 we are dealing with, why not simply fit in the flavour basis and force positivity on the result?