[TIR] cast disparate floating point types for binary ops

[AndrewZhaoLuo]

Right now if we in TIR add a floating point type with an integer type, the integer type will be implicitly cast to the floating point type.

E.g. a: float32 + b: int32 ---> a: float32 + cast(float32, b: int32)

This change does the same thing for floating point types. If we have two different floating point types e.g. fp16 and fp32, then when operating, the lower bit floating point type gets cast into the higher bit floating point type.

E.g. a: float32 + b: float16 --> a: float32 + cast(float32, b: float16)

This is of use since #8340 has an issue where some schedules which should support mixed precision types do not. This is due to binary ops like addition and multiplication not supporting mixing fp32 and fp16. Most of these errors can be fixed by inserting a cast into the schedule.

Rather than manually audit every schedule which might have this, this might be a preferable and reasonable solution.

[AndrewZhaoLuo]

Thanks for the review @comaniac PTAL

[AndrewZhaoLuo]

By the way, please don't merge this anytime soon. I want to get some input from some others like @jroesch or @junrushao1994

[comaniac]

LGTM. Leave it to @jroesch @junrushao1994

[jroesch]

As long as we don't implicitly downcast I think this should be fine, can you test behavior of storing both an i32/u32 computation in i16/u16, and the other way around?

[AndrewZhaoLuo]

As long as we don't implicitly downcast I think this should be fine, can you test behavior of storing both an i32/u32 computation in i16/u16, and the other way around?

This will fail since this PR only upcasts at most one arg. in binary operations only (e.g. addition). Do you want me to also upcast assignment for floating point types to be consistent? @jroesch

[jroesch]

As long as we don't implicitly downcast I think this should be fine, can you test behavior of storing both an i32/u32 computation in i16/u16, and the other way around?

This will fail since this PR only upcasts at most one arg. in binary operations only (e.g. addition). Do you want me to also upcast assignment for floating point types to be consistent? @jroesch

I meant we should add negative tests so that if someone later comes along to modify the behavior we have clearly written down what should pass and what should fail.

[AndrewZhaoLuo]

As long as we don't implicitly downcast I think this should be fine, can you test behavior of storing both an i32/u32 computation in i16/u16, and the other way around?

This will fail since this PR only upcasts at most one arg. in binary operations only (e.g. addition). Do you want me to also upcast assignment for floating point types to be consistent? @jroesch

I meant we should add negative tests so that if someone later comes along to modify the behavior we have clearly written down what should pass and what should fail.

Oh ok. Done.

[AndrewZhaoLuo]

PTAL @jroesch

[jroesch]

LGTM, just get it green and we are gtg

[comaniac]

Thanks @AndrewZhaoLuo @jroesch