Unalignment case for conv

[mengchihe]

I find out that I can set AlignmentA/B in gemm to handle the case when shape is unalignment for int4.
But how can I run conv in similar case, such as channel equals to 1 or 3.
Is there any configuration to set global load granularity for conv, thanks.

[hwu36]

CONV currently do not support this feature. However, it is pretty easy to add it.

In GEMM, this line splits a 128-bit load into multiple smaller loads. You just need to do the same to the conv mainloop in https://github.com/NVIDIA/cutlass/blob/master/include/cutlass/conv/threadblock/implicit_gemm_multistage.h . The underlying iterators in https://github.com/NVIDIA/cutlass/blob/master/include/cutlass/conv/threadblock and several defaultxxx files need some plumbing, too.

We welcome the community to upstream this feature.

[mengchihe]

okay thanks, I will try to upload a patch

[hwu36]

Awesome, just first try to figure out how GEMM works, and then just do the same for CONV. It is not hard.