Simplify cat implementation

[naoyam]

cat is translated to CUDA code with a if-then-else block:

if (i < input0_ext) {
  out[idx] = input0[idx];
} else if (i < input0_ext + input1_ext) {
  out[idx] = input1[idx];
} else if ...

This is correct, but I believe this can be simplified to just out[idx] = input0[idx] + input1[idx] + ... since all of the inputs are padded by zero, so this result should be equivalent.

Since + is not defined for some low precision types, bitwise-or is instead used when addition is not available.

On A100, this simplification yielded about 5% perf improvement of a RoPE module.

I didn't add any specific test since I don't know if any new test would be beneficial. Half, bfloat16 and fp8 types are tested by ResizeTest.CatMemoryPromotionReducedFloating.

[naoyam]

Not exactly sure if this is reasonable. I don't know why memcpy is extensively used for these fp8 types.

Pinging @jjsjann123

[jjsjann123]

This does look cleaner :)

[naoyam]

I thought there may be some magic with memcpy.

[naoyam]

!build

[jacobhinkle]

LGTM. I wonder if we could handle all types (other than ComplexDouble) by bitcasting to an integer of the right size then doing bitwise or. For this we would need Int8 and Int16, but we would not need support in the runtime files for bitwise ops on floats then.

[naoyam]

LGTM. I wonder if we could handle all types (other than ComplexDouble) by bitcasting to an integer of the right size then doing bitwise or. For this we would need Int8 and Int16, but we would not need support in the runtime files for bitwise ops on floats then.

Yeah, I did think about it, but I found it's just easier to handle these low precision types separately than adding the new integer types, at least for now.