Rotate the eko back to save disk space

[andreab1997]

We want to rotate the eko back to the usual grid of 50 points to save disk space. In fact, as noticed by @giacomomagni, depending on the dataset, there is a factor up to ~30 in size between the original eko and the eko after the rotation (for the jets dataset the factor is ~24).

[alecandido]

Please, avoid rotating back. Just keep the original one (copy/duplicate it before).

Otherwise, you will accumulate more and more numerical error.

[andreab1997]

Please, avoid rotating back. Just keep the original one (copy/duplicate it before).

Otherwise, you will accumulate more and more numerical error.

Yes this was exactly my point. However @giacomomagni pointed out that copying 60gb every time (for the jets ekos) is a bit extreme.

[alecandido]

Yes this was exactly my point. However @giacomomagni pointed out that copying 60gb every time (for the jets ekos) is a bit extreme.

If you do not need the original one, then you can well thrash it, and save space. But if you need it, then instead you should keep.
Notice that there is less computation involved in the copy, rather than going back and forth, so it's not a problem of performances. If it's a problem of disk space, you cancel the rotated EKOs right after. If you have enough space, and you want to save computation, you keep both.

The only case which we can not account for is if you do not have enough space to even perform the operation (copying), and you want to keep the original EKO.
In this case, the optimal solution would be to rotate on the fly, so we should give back an iterator from manipulate, that for each value of the scale: loads the corresponding array, rotates it in memory, returns the result without saving it. I.e., if this is a concern, we need a lazy iterator.

[felixhekhorn]

In this case, the optimal solution would be to rotate on the fly, so we should give back an iterator from manipulate, that for each value of the scale: loads the corresponding array, rotates it in memory, returns the result without saving it. I.e., if this is a concern, we need a lazy iterator.

Maybe we should start rolling out eko v0.14 with NNPDF/eko#295 and specifically NNPDF/eko#292

[giacomomagni]

I'm try to see if this temporary 5016b86 solution is doable also for jets.

[giacomomagni]

Thanks @andreab1997 this solves the issue.
I'd be happy to merge.

[andreab1997]

Thanks @andreab1997 this solves the issue. I'd be happy to merge.

I would be happy to merge as well now.