Add a multidimensional matrix multiply

[bluss]

General dimensions, like numpy::dot.

Restricted to float types, for transparent blas support.

[termoshtt]

NumPy has einsum (Einstein Summation) for effective higher order tensor manipulation.
https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html
I think it is better to implement general einsum for rust-ndarray, and rewrite dot to call specific einsum.
I hope the following pseudo-code works:

let a = Array::<f64, _>::random((7, 5, 10));
let b = Array::<f64, _>::random((10, 5));
let c = einsum!("ijk", "kj" -> "i" | a, b); // results one-dim array with 7 components

[bluss]

Then I'd be curious how you recover the performance of matrix multiplication. ndarray uses crate matrixmultiply which uses a packing strategy and a vectorized kernel.

[termoshtt]

I saw opt-numpy, which decompose given einsum into more fundamental tensor reduction. This project can detects the case where dot or dgemm can be used, and in this case the performance will be good. I have no idea for another case about calculation performance :<

[bluss]

Thanks for the link. I'd spontaneously say that a good einsum is a project as large as ndarray itself (almost) and would be a thing to build in its own crate.

[ngoldbaum]

Would be nice to have more matrix multiply implementations. Even expanding the set of implemented types to include Array3 would be helpful for me, it's currently not straightforward to port numpy code that does vectorized matrix multiplies (e.g. matrix multiplication on stacks of matrices or vectors).

[chenpeizhi]

The TACO project looks very promising. It was designed with both dense and sparse tensors in mind, supporting various formats of sparse tensors and block sparse tensors. Its API is similar to Julia TensorOperations or Nim Arraymancer, which is more intuitive than that of NumPy einsum or opt_einsum. I wonder whether there is any plan to interface with TACO or implement similar functionalities.