gfx908 optimizations

[IMbackK]

This minor optimization work increases CDNA performance by around 10x.

Current master:

model	size	params	backend	ngl	test	t/s
llama 70B Q4_K - Medium	39.59 GiB	70.55 B	ROCm	99	pp1024	56.73 ± 0.34
llama 7B Q4_K - Medium	4.07 GiB	7.24 B	ROCm	99	pp1024	585.58 ± 0.86

This pr:

model	size	params	backend	ngl	test	t/s
llama 70B Q4_K - Medium	39.59 GiB	70.55 B	ROCm	99	pp1024	462.30 ± 0.95
llama 7B Q4_K - Medium	4.07 GiB	7.24 B	ROCm	99	pp1024	3196.68 ± 1.97

As now most of the of the remaining time is spent in attn kernels, merging #7011 further increases performance by 2x

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[JohannesGaessler]

Which specific GPU are you using?

[IMbackK]

gfx908 aka MI100, gfx90a aka mi200 family should have completely identical performance characteristics.

[ccbadd]

If this gets merged I'm going to have to fire up my server with the 2X MI100s and give it another try. I never understood why they were pretty much the same speed as my W6800 Pro's.

[IMbackK]

If this gets merged I'm going to have to fire up my server with the 2X MI100s and give it another try. I never understood why they were pretty much the same speed as my W6800 Pro's.

Well this ofc dosent help at all with token generation as gemv is the most time consumeing kenrel thair. In generall looking at omniperf, there is quite some distance to go there for decent performance

[JohannesGaessler]

There is some parallel work by me on the matrix multiplication code in #8062 and #8075 that touch the same code as this PR. After these two PRs have been merged, please adapt your changes and confirm that the performance improvement is still there. Sorry for the inconvenience.

[IMbackK]

closed due to obsoletion for now, i will reopen with a rebased version at some point in the future