Skip to content

Overflow in colsums #4

@privefl

Description

@privefl

Hello,

I personally use a big.matrix of 0, 1, 2 only. So I use type char.
If I make no mistake, the maximum char is 127.

So when using this function on a big.matrix (so, with a large number of rows)

Map<MatrixXchar> bM = Map<MatrixXchar>((char *)xpMat->matrix(), xpMat->nrow(), xpMat->ncol());
Vectorchar colSums = bM.colwise().sum();

it will results in an overflow (the results will be between -128 and 127 whereas it should be more than 10,000 maybe).

Would it be possible (and more pertinent) to replace Vectorchar colSums by VectorXi colSums for char and short types?

Anyway, thanks for introducing me to Mapping big.matrix objects in Eigen.

Florian

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions