CUDA: fuse rope + set_rows

[am17an]

Based on #16769.

On a 4090:

Model	Test	t/s master	t/s cuda-rope-fusion	Speedup
llama 8B Q4_K_M	tg32	134.90	136.07	1.01
llama 8B Q4_K_M	tg64	131.41	132.84	1.01
llama 8B Q4_K_M	tg128	130.54	131.87	1.01
qwen3moe 30B.A3B Q4_0	tg32	167.18	168.23	1.01
qwen3moe 30B.A3B Q4_0	tg64	161.00	161.90	1.01
qwen3moe 30B.A3B Q4_0	tg128	158.84	159.83	1.01

[ORippler]

While the fusion itself is quite simple, I would still recommend to add a test to test-backend-ops for it nonetheless

[am17an]

While the fusion itself is quite simple, I would still recommend to add a test to test-backend-ops for it nonetheless

There is already a test added in the vulkan PR #16769

[JohannesGaessler]

I forgot: this is the performance I measured. The reordering of the ggml graph seems to be very slightly slower in some cases but as this is difficult to generalize I think it's fine.

GPU	Model	Microbatch size	Test	t/s 655cddd	t/s 6762493	Speedup
MI60 / MI50	llama 8B Q4_0	1	pp512	104.58	103.97	0.99
MI60 / MI50	llama 8B Q4_0	2	pp512	178.03	175.58	0.99
MI60 / MI50	llama 8B Q4_0	4	pp512	167.55	166.49	0.99
MI60 / MI50	llama 8B Q4_0	8	pp512	226.62	225.66	1.00
MI60 / MI50	llama 8B Q4_0	16	pp512	370.58	370.28	1.00
MI60 / MI50	llama 8B Q4_0	32	pp512	478.61	478.51	1.00
MI60 / MI50	llama 8B Q4_0	64	pp512	551.56	550.08	1.00
MI60 / MI50	llama 8B Q4_0	128	pp512	761.23	766.59	1.01
MI60 / MI50	llama 8B Q4_0	256	pp512	908.92	908.78	1.00
MI60 / MI50	llama 8B Q4_0	512	pp512	1071.50	1071.73	1.00
MI100	llama 8B Q4_0	1	pp512	131.54	129.94	0.99
MI100	llama 8B Q4_0	2	pp512	205.84	209.33	1.02
MI100	llama 8B Q4_0	4	pp512	221.23	221.47	1.00
MI100	llama 8B Q4_0	8	pp512	313.47	312.99	1.00
MI100	llama 8B Q4_0	16	pp512	750.72	733.09	0.98
MI100	llama 8B Q4_0	32	pp512	1221.46	1204.27	0.99
MI100	llama 8B Q4_0	64	pp512	1714.07	1656.16	0.97
MI100	llama 8B Q4_0	128	pp512	1891.37	1855.71	0.98
MI100	llama 8B Q4_0	256	pp512	2242.31	2233.38	1.00
MI100	llama 8B Q4_0	512	pp512	2353.38	2319.60	0.99
P40	llama 8B Q4_0	1	pp512	59.09	58.97	1.00
P40	llama 8B Q4_0	2	pp512	115.32	115.23	1.00
P40	llama 8B Q4_0	4	pp512	159.03	159.38	1.00
P40	llama 8B Q4_0	8	pp512	217.42	217.87	1.00
P40	llama 8B Q4_0	16	pp512	473.44	474.34	1.00
P40	llama 8B Q4_0	32	pp512	571.08	571.67	1.00
P40	llama 8B Q4_0	64	pp512	777.03	777.95	1.00
P40	llama 8B Q4_0	128	pp512	891.91	892.64	1.00
P40	llama 8B Q4_0	256	pp512	980.57	980.75	1.00
P40	llama 8B Q4_0	512	pp512	1017.64	1018.90	1.00
RTX 3090	llama 8B Q4_0	1	pp512	167.94	168.12	1.00
RTX 3090	llama 8B Q4_0	2	pp512	289.07	289.16	1.00
RTX 3090	llama 8B Q4_0	4	pp512	480.81	480.64	1.00
RTX 3090	llama 8B Q4_0	8	pp512	598.67	595.97	1.00
RTX 3090	llama 8B Q4_0	16	pp512	1263.01	1243.59	0.98
RTX 3090	llama 8B Q4_0	32	pp512	2102.57	2083.90	0.99
RTX 3090	llama 8B Q4_0	64	pp512	3265.39	3278.28	1.00
RTX 3090	llama 8B Q4_0	128	pp512	4271.68	4254.93	1.00
RTX 3090	llama 8B Q4_0	256	pp512	5019.34	5011.89	1.00
RTX 3090	llama 8B Q4_0	512	pp512	5322.35	5303.43	1.00
RTX 4090	llama 8B Q4_0	1	pp512	196.78	197.77	1.01
RTX 4090	llama 8B Q4_0	2	pp512	336.20	338.98	1.01
RTX 4090	llama 8B Q4_0	4	pp512	656.65	660.65	1.01
RTX 4090	llama 8B Q4_0	8	pp512	1084.41	1088.33	1.00
RTX 4090	llama 8B Q4_0	16	pp512	1827.57	1834.48	1.00
RTX 4090	llama 8B Q4_0	32	pp512	3338.89	3348.81	1.00
RTX 4090	llama 8B Q4_0	64	pp512	5797.79	5807.68	1.00
RTX 4090	llama 8B Q4_0	128	pp512	8651.31	8664.61	1.00
RTX 4090	llama 8B Q4_0	256	pp512	11456.76	11489.10	1.00
RTX 4090	llama 8B Q4_0	512	pp512	12896.69	12850.92	1.00
RTX 5090	llama 8B Q4_0	1	pp512	301.86	303.81	1.01
RTX 5090	llama 8B Q4_0	2	pp512	430.28	436.76	1.02
RTX 5090	llama 8B Q4_0	4	pp512	808.19	820.35	1.02
RTX 5090	llama 8B Q4_0	8	pp512	1205.88	1217.64	1.01
RTX 5090	llama 8B Q4_0	16	pp512	2021.53	2036.90	1.01
RTX 5090	llama 8B Q4_0	32	pp512	3628.93	3629.65	1.00
RTX 5090	llama 8B Q4_0	64	pp512	5802.86	5797.75	1.00
RTX 5090	llama 8B Q4_0	128	pp512	8009.47	7984.37	1.00
RTX 5090	llama 8B Q4_0	256	pp512	11673.97	11751.83	1.01
RTX 5090	llama 8B Q4_0	512	pp512	14647.57	14691.56	1.00
RX 6800	llama 8B Q4_0	1	pp512	76.98	77.58	1.01
RX 6800	llama 8B Q4_0	2	pp512	138.23	138.67	1.00
RX 6800	llama 8B Q4_0	4	pp512	194.59	195.50	1.00
RX 6800	llama 8B Q4_0	8	pp512	224.51	224.57	1.00
RX 6800	llama 8B Q4_0	16	pp512	315.48	316.02	1.00
RX 6800	llama 8B Q4_0	32	pp512	468.10	469.08	1.00
RX 6800	llama 8B Q4_0	64	pp512	591.79	592.87	1.00
RX 6800	llama 8B Q4_0	128	pp512	758.91	759.77	1.00
RX 6800	llama 8B Q4_0	256	pp512	887.36	888.37	1.00
RX 6800	llama 8B Q4_0	512	pp512	954.53	956.51	1.00
RX 9060 XT	llama 8B Q4_0	1	pp512	49.14	49.65	1.01
RX 9060 XT	llama 8B Q4_0	2	pp512	94.57	95.03	1.00
RX 9060 XT	llama 8B Q4_0	4	pp512	175.62	176.57	1.01
RX 9060 XT	llama 8B Q4_0	8	pp512	205.97	206.70	1.00
RX 9060 XT	llama 8B Q4_0	16	pp512	505.09	507.65	1.01
RX 9060 XT	llama 8B Q4_0	32	pp512	650.56	670.85	1.03
RX 9060 XT	llama 8B Q4_0	64	pp512	177.49	178.31	1.00
RX 9060 XT	llama 8B Q4_0	128	pp512	317.85	318.09	1.00
RX 9060 XT	llama 8B Q4_0	256	pp512	341.85	341.61	1.00
RX 9060 XT	llama 8B Q4_0	512	pp512	354.91	354.45	1.00
V100-PCIE-32GB	llama 8B Q4_0	1	pp512	138.71	139.97	1.01
V100-PCIE-32GB	llama 8B Q4_0	2	pp512	252.68	254.78	1.01
V100-PCIE-32GB	llama 8B Q4_0	4	pp512	345.40	347.20	1.01
V100-PCIE-32GB	llama 8B Q4_0	8	pp512	522.47	525.07	1.00
V100-PCIE-32GB	llama 8B Q4_0	16	pp512	788.52	791.32	1.00
V100-PCIE-32GB	llama 8B Q4_0	32	pp512	1177.44	1182.43	1.00
V100-PCIE-32GB	llama 8B Q4_0	64	pp512	629.38	629.72	1.00
V100-PCIE-32GB	llama 8B Q4_0	128	pp512	1185.78	1186.15	1.00
V100-PCIE-32GB	llama 8B Q4_0	256	pp512	1970.08	1970.74	1.00
V100-PCIE-32GB	llama 8B Q4_0	512	pp512	2699.19	2701.41	1.00

[am17an]

Ok just to be sure I ran PPL tests with and without this PR and they look all good with/without -ot