Optimize all subgrids

[cschwan]

Right now ImportOnlySubgridV1 isn't optimized, as are other grids; these are supposedly already optimized. However, with #193 merged all grid types must be considered for optimization:

pineappl/pineappl/src/grid.rs

Lines 1105 to 1120 in 86d9a72

	match subgrid {
	SubgridEnum::LagrangeSubgridV1(grid) => {
	let mut new_subgrid = LagrangeSparseSubgridV1::from(&*grid).into();
	mem::swap(subgrid, &mut new_subgrid);
	}
	SubgridEnum::LagrangeSubgridV2(grid) => {
	let mut new_subgrid = ImportOnlySubgridV2::from(&*grid).into();
	mem::swap(subgrid, &mut new_subgrid);
	}
	SubgridEnum::EmptySubgridV1(_)
	| SubgridEnum::LagrangeSparseSubgridV1(_)
	| SubgridEnum::ImportOnlySubgridV1(_)
	| SubgridEnum::ImportOnlySubgridV2(_) => {
	// nothing to optimize here
	}
	SubgridEnum::NtupleSubgridV1(_) => todo!(),

With commit f79b9c6 we finally have a general interface to do this: Subgrid::indexed_iter.

[cschwan]

This has been implemented in commit 399eae7 and fixes have been added with commit c8d0dfa.

[cschwan]

The has a relatively large impact on the size of uncompressed grids, but is completely irrelevant for compressed grids (LZ4 does its job quite well):

ATLAS_DY_7TEV_49FB_HIMASS              - reduction of 15.96% /  0.16% (compressed)
ATLAS_DY_7TEV_CC                       - reduction of 20.01% /  0.35% (compressed)
ATLAS_DY_7TEV_LOMASS_EXT               - reduction of  7.58% /  0.36% (compressed)
ATLAS_TTB_13TEV_TOT                    - reduction of  8.58% /  0.41% (compressed)
ATLAS_TTB_7TEV_TOT                     - reduction of 11.32% /  0.58% (compressed)
ATLAS_TTB_8TEV_2L_TTM                  - reduction of 13.11% /  0.44% (compressed)
ATLAS_TTB_8TEV_2L_TTRAP                - reduction of 15.68% /  0.47% (compressed)
ATLAS_TTB_8TEV_LJ_TPT                  - reduction of 10.43% /  0.10% (compressed)
ATLAS_TTB_8TEV_LJ_TRAP                 - reduction of 13.45% /  0.71% (compressed)
ATLAS_TTB_8TEV_LJ_TTM                  - reduction of 13.04% /  0.38% (compressed)
ATLAS_TTB_8TEV_LJ_TTRAP                - reduction of 16.18% /  0.42% (compressed)
ATLAS_TTB_8TEV_TOT                     - reduction of 10.65% /  0.52% (compressed)
ATLAS_WM_7TEV                          - reduction of  7.81% / -0.00% (compressed)
ATLAS_WP_7TEV                          - reduction of  8.27% /  0.04% (compressed)
CMS_DY_13TEV_28FB                      - reduction of 12.94% /  0.19% (compressed)
CMS_DY_14TEV_MLL_5000_COSTH            - reduction of 30.05% /  0.30% (compressed)
CMS_TTB_13TEV_2L_TPT                   - reduction of  8.14% /  0.09% (compressed)
CMS_TTB_13TEV_2L_TRAP                  - reduction of 12.11% /  0.54% (compressed)
CMS_TTB_13TEV_2L_TTM                   - reduction of 11.37% /  0.45% (compressed)
CMS_TTB_13TEV_2L_TTRAP                 - reduction of 14.82% /  0.62% (compressed)
CMS_TTB_13TEV_LJ_TPT                   - reduction of  8.76% /  0.10% (compressed)
CMS_TTB_13TEV_LJ_TRAP                  - reduction of 12.87% /  0.57% (compressed)
CMS_TTB_13TEV_LJ_TTM                   - reduction of 10.98% /  0.48% (compressed)
CMS_TTB_13TEV_LJ_TTRAP                 - reduction of 15.96% /  0.64% (compressed)
CMS_TTB_5TEV_TOT                       - reduction of 13.30% /  0.66% (compressed)
CMS_ZPT_13TEV_358FB                    - reduction of  7.72% /  0.15% (compressed)
D0WMASY-grid-40-6-15-3-Wminus_wly_pt25 - reduction of  1.39% /  0.23% (compressed)
D0WMASY-grid-40-6-15-3-Wplus_wly_pt25  - reduction of  1.46% / -1.33% (compressed)
LHCB_DY_13TEV_DIELECTRON               - reduction of 16.17% /  0.45% (compressed)
LHCB_DY_13TEV_DIMUON                   - reduction of 16.03% /  0.45% (compressed)
LHCB_DY_7TEV_940PB                     - reduction of 15.97% /  0.41% (compressed)
LHCB_DY_7TEV                           - reduction of 17.29% /  0.44% (compressed)
LHCB_DY_8TEV_2FB                       - reduction of 16.80% /  0.45% (compressed)
LHCB_DY_8TEV                           - reduction of 16.84% /  0.44% (compressed)
LHCB_WM_7TEV                           - reduction of  4.38% / -0.08% (compressed)
LHCB_WM_8TEV                           - reduction of  4.36% / -0.09% (compressed)
LHCB_WP_7TEV                           - reduction of  5.05% / -0.03% (compressed)
LHCB_WP_8TEV                           - reduction of  4.98% / -0.03% (compressed)

[alecandido]

The has a relatively large impact on the size of uncompressed grids,

But this still means we're gaining in memory in performance, even though not in storage.

[cschwan]

But this still means we're gaining in memory in performance, even though not in storage.

Absolutely, and the reduction in memory should be 1:1 as shown. The biggest advantage should be in the evolution, however, because the slicing of the EKOs in pineappl::evolution::operators should produce smaller matrices. I should verify that though.