12/04 Make things done

[5ujinKang]

• Algorithm itself doesn't have a problem : seems
• somehow if we use CANN function to implement, aclnn_convolution . mayve cubeMathType param is the problem. / maybe can use FP16 types( https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/aolapi/context/aclnnConvolution.md )

MAKE IT WORKS

[0Marble]

• Possible fix if it is the f16: in tests/test-backend-ops.cpp, the struct test_ssm_conv test inherits from struct test_case, which has the max_nmse_err() set to 1e-7.
• On struct test_mul_mat, the method is overloaded to be 5e-4, which possibly allows the test to pass even tho it is also running on f16.
• Solution 1: just overload the method to have a more loose error.
• We will need to try to set mat_mul to have the tight error, and make sure that the test doesnt pass on f32 inputs.
• That way we can prove that the cause of the error is precisely the f16 problem.

[0Marble]

• Secondary issue: wasteful implementation
• Right now, we have to change the shape of W from [d_inner, d_conv] to [d_inner, d_inner, d_conv], and d_inner is a big dimension (1024, 2048, ...).
• The solution also requires a bunch of extra operations to get the W matrix to be diagonal (create identity matrix, make it 3d, stack Ws, multiply to get W on the diagonal).
• I dont know if we can avoid this without an actual custom kernel