Redefine channels such that the number of non-zero subgrids is minimal

[cschwan]

Hawaiian Vrap produces luminosities that

• contain lots of entries with with different numerical factors,
• entries that overlap a lot and finally
• each bin has its own luminosities.

Here's an example for the first bin of DYE866R_P, all other bins have the same tuples, but with different factors:

 l                 entry                               entry                                entry                               entry                               entry                                entry                               entry                                entry                               entry                                entry                               entry                                entry                                entry                               entry                                entry                               entry                                entry                               entry                                entry                               entry                               entry                                entry                               entry                                entry                               entry                               entry                                entry                               entry                               entry                                entry                               entry                                entry                               entry                                entry                               entry                                entry                                entry                               entry                                entry                               entry                                entry                               entry                                entry                               entry                               entry                                entry                               entry                                entry                               entry                                entry                               entry                                entry                                entry                               entry                               entry                               entry                               entry                               entry                                entry                               entry                               entry                                entry                               entry                              entry
---+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+-----------------------------------+------------------------------------+------------------------------------+-----------------------------------+------------------------------------+----------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+------------------------------------+-----------------------------------+-----------------------------------+----------------------------------
0   0.1110816720085171 × (-3,  3)       0.44441143407799993 × (-2,  2)       0.1110816720085171 × (-1,  1)       0.1110816720085171 × ( 1, -1)       0.44441143407799993 × ( 2, -2)       0.1110816720085171 × ( 3, -3)       0.8888228681559999 × ( 4,  4)        0.2221633440170342 × ( 5,  5)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
1   0.1110816720085171 × (-3, 21)       0.44441143407799993 × (-2, 21)       0.1110816720085171 × (-1, 21)       0.1110816720085171 × ( 1, 21)       0.44441143407799993 × ( 2, 21)       0.1110816720085171 × ( 3, 21)       0.8888228681559999 × ( 4, 21)        0.2221633440170342 × ( 5, 21)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
2   0.0000008334356430357662 × (-3,  3) -0.0000008334356430357662 × (-2,  2) 0.0000008334356430357662 × (-1,  1) 0.0000008334356430357662 × ( 1, -1) -0.0000008334356430357662 × ( 2, -2) 0.0000008334356430357662 × ( 3, -3) -0.0000016668712860715325 × ( 4,  4) 0.0000016668712860715325 × ( 5,  5)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
3   0.2221633440170342 × (-3,  3)       0.8888228681559999 × (-2,  2)        0.2221633440170342 × (-1,  1)       0.2221633440170342 × ( 1, -1)       0.8888228681559999 × ( 2, -2)        0.2221633440170342 × ( 3, -3)       1.7776457363119997 × ( 4,  4)        0.4443266880340684 × ( 5,  5)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
4   1.222067884181551 × (-3,  3)        1.222067884181551 × (-2,  2)         1.222067884181551 × (-1,  1)        1.222067884181551 × ( 1, -1)        1.222067884181551 × ( 2, -2)         1.222067884181551 × ( 3, -3)        2.444135768363102 × ( 4,  4)         2.444135768363102 × ( 5,  5)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
5   0.1110816720085171 × (-3, -3)       0.1110816720085171 × (-3, -2)        0.1110816720085171 × (-3, -1)       0.1110816720085171 × (-3,  1)       0.1110816720085171 × (-3,  2)        0.1110816720085171 × (-3,  3)       0.2221633440170342 × (-3,  4)        0.2221633440170342 × (-3,  5)       0.44441143407799993 × (-2, -3)       0.44441143407799993 × (-2, -2)      0.44441143407799993 × (-2, -1)       0.44441143407799993 × (-2,  1)       0.44441143407799993 × (-2,  2)      0.44441143407799993 × (-2,  3)       0.8888228681559999 × (-2,  4)       0.8888228681559999 × (-2,  5)        0.1110816720085171 × (-1, -3)       0.1110816720085171 × (-1, -2)        0.1110816720085171 × (-1, -1)       0.1110816720085171 × (-1,  1)       0.1110816720085171 × (-1,  2)        0.1110816720085171 × (-1,  3)       0.2221633440170342 × (-1,  4)        0.2221633440170342 × (-1,  5)       0.1110816720085171 × ( 1, -3)       0.1110816720085171 × ( 1, -2)        0.1110816720085171 × ( 1, -1)       0.1110816720085171 × ( 1,  1)       0.1110816720085171 × ( 1,  2)        0.1110816720085171 × ( 1,  3)       0.2221633440170342 × ( 1,  4)        0.2221633440170342 × ( 1,  5)       0.44441143407799993 × ( 2, -3)       0.44441143407799993 × ( 2, -2)      0.44441143407799993 × ( 2, -1)       0.44441143407799993 × ( 2,  1)       0.44441143407799993 × ( 2,  2)      0.44441143407799993 × ( 2,  3)       0.8888228681559999 × ( 2,  4)       0.8888228681559999 × ( 2,  5)        0.1110816720085171 × ( 3, -3)       0.1110816720085171 × ( 3, -2)        0.1110816720085171 × ( 3, -1)       0.1110816720085171 × ( 3,  1)       0.1110816720085171 × ( 3,  2)        0.1110816720085171 × ( 3,  3)       0.2221633440170342 × ( 3,  4)        0.2221633440170342 × ( 3,  5)       0.8888228681559999 × ( 4, -3)        0.8888228681559999 × ( 4, -2)       0.8888228681559999 × ( 4, -1)        0.8888228681559999 × ( 4,  1)        0.8888228681559999 × ( 4,  2)       0.8888228681559999 × ( 4,  3)        1.7776457363119997 × ( 4,  4)      1.7776457363119997 × ( 4,  5)       0.2221633440170342 × ( 5, -3)       0.2221633440170342 × ( 5, -2)        0.2221633440170342 × ( 5, -1)       0.2221633440170342 × ( 5,  1)       0.2221633440170342 × ( 5,  2)        0.2221633440170342 × ( 5,  3)       0.4443266880340684 × ( 5,  4)       0.4443266880340684 × ( 5,  5)
6   -0.44441060064235693 × (-2, -2)     0.22218328037455254 × (-2, -1)       -0.22218328037455254 × (-2,  1)     0.44441060064235693 × (-2,  2)      0.22218328037455254 × (-1, -2)       -0.11108083857287405 × (-1, -1)     0.11108083857287405 × (-1,  1)       -0.22218328037455254 × (-1,  2)     -0.22218328037455254 × ( 1, -2)      0.11108083857287405 × ( 1, -1)      -0.11108083857287405 × ( 1,  1)      0.22218328037455254 × ( 1,  2)       0.44441060064235693 × ( 2, -2)      -0.22218328037455254 × ( 2, -1)      0.22218328037455254 × ( 2,  1)      -0.44441060064235693 × ( 2,  2)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
7   0.0000008334356430357662 × (-3, -3) -0.0000008334356430357662 × (-3, -2) 0.0000008334356430357662 × (-3, -1) 0.0000008334356430357662 × (-3,  1) -0.0000008334356430357662 × (-3,  2) 0.0000008334356430357662 × (-3,  3) -0.0000016668712860715325 × (-3,  4) 0.0000016668712860715325 × (-3,  5) -0.0000008334356430357662 × (-2, -3) 0.0000008334356430357662 × (-2, -2) -0.0000008334356430357662 × (-2, -1) -0.0000008334356430357662 × (-2,  1) 0.0000008334356430357662 × (-2,  2) -0.0000008334356430357662 × (-2,  3) 0.0000016668712860715325 × (-2,  4) -0.0000016668712860715325 × (-2,  5) 0.0000008334356430357662 × (-1, -3) -0.0000008334356430357662 × (-1, -2) 0.0000008334356430357662 × (-1, -1) 0.0000008334356430357662 × (-1,  1) -0.0000008334356430357662 × (-1,  2) 0.0000008334356430357662 × (-1,  3) -0.0000016668712860715325 × (-1,  4) 0.0000016668712860715325 × (-1,  5) 0.0000008334356430357662 × ( 1, -3) -0.0000008334356430357662 × ( 1, -2) 0.0000008334356430357662 × ( 1, -1) 0.0000008334356430357662 × ( 1,  1) -0.0000008334356430357662 × ( 1,  2) 0.0000008334356430357662 × ( 1,  3) -0.0000016668712860715325 × ( 1,  4) 0.0000016668712860715325 × ( 1,  5) -0.0000008334356430357662 × ( 2, -3) 0.0000008334356430357662 × ( 2, -2) -0.0000008334356430357662 × ( 2, -1) -0.0000008334356430357662 × ( 2,  1) 0.0000008334356430357662 × ( 2,  2) -0.0000008334356430357662 × ( 2,  3) 0.0000016668712860715325 × ( 2,  4) -0.0000016668712860715325 × ( 2,  5) 0.0000008334356430357662 × ( 3, -3) -0.0000008334356430357662 × ( 3, -2) 0.0000008334356430357662 × ( 3, -1) 0.0000008334356430357662 × ( 3,  1) -0.0000008334356430357662 × ( 3,  2) 0.0000008334356430357662 × ( 3,  3) -0.0000016668712860715325 × ( 3,  4) 0.0000016668712860715325 × ( 3,  5) -0.0000016668712860715325 × ( 4, -3) 0.0000016668712860715325 × ( 4, -2) -0.0000016668712860715325 × ( 4, -1) -0.0000016668712860715325 × ( 4,  1) 0.0000016668712860715325 × ( 4,  2) -0.0000016668712860715325 × ( 4,  3) 0.000003333742572143065 × ( 4,  4) -0.000003333742572143065 × ( 4,  5) 0.0000016668712860715325 × ( 5, -3) -0.0000016668712860715325 × ( 5, -2) 0.0000016668712860715325 × ( 5, -1) 0.0000016668712860715325 × ( 5,  1) -0.0000016668712860715325 × ( 5,  2) 0.0000016668712860715325 × ( 5,  3) -0.000003333742572143065 × ( 5,  4) 0.000003333742572143065 × ( 5,  5)
8   0.1110816720085171 × (-3, -3)       0.44441143407799993 × (-2, -2)       0.1110816720085171 × (-1, -1)       0.1110816720085171 × ( 1,  1)       0.44441143407799993 × ( 2,  2)       0.1110816720085171 × ( 3,  3)       0.8888228681559999 × ( 4,  4)        0.2221633440170342 × ( 5,  5)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
9   1.222067884181551 × (21, 21)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           

One strategy to make the luminosity function more readable and to optimize it is to

1. decompose it first into the entries with the same factors (that would break up l = 0 into four entries)
2. burn the factors into the subgrids
3. merge same entries
4. if that doesn't improve the number of luminosities undo the last steps.

[cschwan]

Yesterday I realized that this problem is actually one of linear (in)dependence and therefore one can frame this as linear algebra problem.

Imagine we have a grid with only one bin and one order, only consisting of $n$ subgrids $g_j$, where $j \in { 1, 2, \ldots, n }$ denotes the channel (index). We then usually have $m$ (non-zero) PDF combinations labelled with $i \in { 1, 2, \ldots, m }$. To give an example of the PDF combinations we have $b_1 = f_\mathrm{u} (x_1, Q^2) f_{\bar{\mathrm{u}}} (x_2, Q^2)$ and $b_2 = f_\mathrm{c} (x_1, Q^2) f_{\bar{\mathrm{c}}} (x_2, Q^2)$, for instance. Note that usually we have $m > n$, because we can often merge PDF combinations together when they have similar matrix elements.

A convolution then can be written as

$$ \int_0^1 \mathrm{d} x_1 \int_0^1 \mathrm{d} x_2 \int_{Q_\mathrm{min}^2}^{Q_\mathrm{max}^2} \mathrm{d} Q^2 G (x_1, x_2, Q^2) $$

where

$$ G (x_1, x_2, Q^2) = \sum_{i=1}^m \sum_{j=1}^n b_i A_{ij} g_j $$

is the important bit. The objects $b = (b_1, b_2, \ldots, b_m)$ and $g = (g_1, g_2, \ldots, g_n)$ are vectors, i.e. elements of a vector space. $(A_{ij})_{i,j=1}^{m,n}$ is a real and in general non-square $m \times n$ matrix. We can then write the important bit as

$$ G (x_1, x_2, Q^2) = b^T A g $$

In essense, this just transforms the (usually) flavour basis contained expressed by $b$ to a different basis, which is a linear combination of the former.

We can now reformulate the problem in terms of linear algebra: find a decomposition $A = L R$ such that the matrix $R$ has the smallest number of non-zero rows. This will reduce the number subgrids.

I believe this can be done by using a (rank-revealing?) LU decomposition with partial pivoting. This would decompose $A = P^T L U$ so that we have

$$ G = b^T P^T L U g = b^T (P^T L) (U g) = (b')^T A' g'$$

where $g' = U g$ are the new subgrids, which are a linear combination of the old subgrids $g$, and the new lumi definition $A' = P^T L$.

[cschwan]

Having read about the CR factorization (alternatively read the preprint) I believe that is what we want, instead of an LU factorization.

We can swap the positions of the grids $g$ and the flavour basis $b$ and have (with differently defined channel matrix $A$):

$$ G = g^T A b = g^T C R b = (C^T g)^T R b = g'^T A' b'$$

with

• $g' = C^T g$, where $C$ is $n \times r$ matrix that contains $r$ independent columns of $A$,
• $A' = R$ where $R$ a $r \times m$ matrix and
• $b' = b$.

Since $r$ is the rank of matrix $A$ and $r \le \mathrm{min}(m,n)$, we have reduced the number of grids needed from $n$ to $r$. Furthermore, $R$ has a simpler structure of the form

$$R = \begin{pmatrix} I_{r \times r} & F_{r \times (n-r)} \end{pmatrix} P_{n \times n}$$

where $P$ is a permutation matrix that just reorders the positions of the flavours in $b$, $I$ is the identity matrix, and $F$ is the matrix that restores the $n-r$ linearly dependent columns not contained in $C$.