[VL][CI] Use a unified docker image for Spark tests

[philo-he]

What changes were proposed in this pull request?

In #7789, Spark-3.4.4 support is added and JDK-17 docker is used to avoid some issues caused by arrow. This pr changes to use the unified docker image and switches JDK version before running the test, which can reduce the efforts a bit for maintaining docker images.

I cannot figure out the reason why this way didn't pass in previous test. Maybe, we can land this pr firstly and continue observing the CI feedback while still keeping the JDK-17 docker image available in docker hub for some time.

How was this patch tested?

CI.

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[zhouyuan]

+1
I remember there is some in-compatible arrow class reported previously. If all tests are passing now so should be all good

[GlutenPerfBot]

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query	log/native_master_03_18_2025_time.csv	log/native_master_03_17_2025_dc6e712845_time.csv	difference	percentage
q1	12.29	11.27	-1.017	91.73%
q2	11.60	11.49	-0.107	99.08%
q3	3.16	3.26	0.099	103.12%
q4	53.66	53.70	0.036	100.07%
q5	10.18	9.15	-1.027	89.92%
q6	4.55	3.93	-0.622	86.32%
q7	5.59	5.37	-0.222	96.04%
q8	4.60	4.81	0.215	104.68%
q9	14.05	17.08	3.034	121.60%
q10	11.74	13.46	1.724	114.68%
q11	29.99	29.71	-0.272	99.09%
q12	3.32	3.66	0.334	110.05%
q13	7.82	5.93	-1.884	75.90%
q14a	43.82	46.65	2.834	106.47%
q14b	40.96	41.34	0.381	100.93%
q15	2.81	4.09	1.287	145.88%
q16	4.42	5.51	1.091	124.71%
q17	7.06	5.99	-1.067	84.89%
q18	8.52	6.96	-1.558	81.71%
q19	3.53	2.74	-0.786	77.73%
q20	1.86	1.81	-0.050	97.30%
q21	0.80	1.66	0.869	209.19%
q22	3.94	2.70	-1.241	68.51%
q23a	62.22	61.51	-0.706	98.87%
q23b	73.49	72.20	-1.288	98.25%
q24a	68.16	69.27	1.112	101.63%
q24b	67.10	64.45	-2.649	96.05%
q25	5.31	5.27	-0.041	99.24%
q26	2.54	2.08	-0.464	81.73%
q27	2.53	2.34	-0.198	92.19%
q28	16.93	17.43	0.500	102.95%
q29	7.70	7.57	-0.131	98.30%
q30	5.87	5.03	-0.844	85.62%
q31	7.97	7.60	-0.372	95.34%
q32	2.73	1.64	-1.091	60.03%
q33	3.24	4.62	1.375	142.40%
q34	3.92	3.73	-0.193	95.06%
q35	8.41	7.26	-1.144	86.39%
q36	2.23	2.74	0.510	122.82%
q37	2.97	3.78	0.804	127.06%
q38	11.98	11.89	-0.087	99.28%
q39a	4.27	4.63	0.369	108.66%
q39b	3.94	4.19	0.247	106.27%
q40	3.81	3.65	-0.159	95.83%
q41	0.65	0.66	0.016	102.54%
q42	0.93	1.75	0.818	187.77%
q43	2.14	2.00	-0.140	93.45%
q44	5.67	5.72	0.049	100.87%
q45	3.38	3.64	0.264	107.81%
q46	4.52	4.64	0.113	102.50%
q47	10.20	10.11	-0.098	99.04%
q48	3.73	3.18	-0.544	85.40%
q49	5.82	5.51	-0.310	94.68%
q50	17.55	17.36	-0.185	98.94%
q51	8.20	7.40	-0.794	90.32%
q52	0.92	1.09	0.175	119.01%
q53	1.80	2.06	0.251	113.91%
q54	5.54	5.59	0.047	100.85%
q55	2.06	1.02	-1.039	49.47%
q56	4.25	4.00	-0.244	94.25%
q57	7.00	6.88	-0.114	98.37%
q58	3.13	3.02	-0.119	96.21%
q59	4.21	4.65	0.447	110.62%
q60	4.48	4.41	-0.067	98.50%
q61	3.85	4.36	0.510	113.25%
q62	3.55	3.24	-0.307	91.36%
q63	1.55	1.46	-0.089	94.26%
q64	36.16	36.34	0.179	100.50%
q65	11.66	11.53	-0.123	98.94%
q66	3.44	3.13	-0.314	90.86%
q67	56.79	57.61	0.820	101.44%
q68	3.30	3.34	0.037	101.13%
q69	4.51	4.87	0.361	108.01%
q70	6.19	5.69	-0.499	91.94%
q71	4.40	3.96	-0.440	90.01%
q72	21.02	20.87	-0.147	99.30%
q73	2.65	2.64	-0.009	99.66%
q74	17.50	17.61	0.107	100.61%
q75	24.86	23.33	-1.530	93.84%
q76	6.20	6.84	0.636	110.26%
q77	2.90	2.36	-0.539	81.42%
q78	33.66	34.13	0.473	101.41%
q79	3.57	4.14	0.571	116.02%
q80	10.47	10.55	0.079	100.75%
q81	6.37	6.57	0.202	103.17%
q82	5.74	5.33	-0.413	92.81%
q83	1.45	1.39	-0.069	95.26%
q84	2.94	2.64	-0.303	89.71%
q85	6.03	5.72	-0.309	94.87%
q86	1.99	2.24	0.255	112.86%
q87	12.12	11.89	-0.227	98.13%
q88	15.37	15.28	-0.095	99.38%
q89	2.50	1.98	-0.521	79.16%
q90	2.16	2.12	-0.031	98.54%
q91	3.94	4.76	0.826	120.98%
q92	1.75	1.96	0.218	112.50%
q93	23.40	24.24	0.845	103.61%
q94	9.10	8.20	-0.899	90.12%
q9	55.48	55.09	-0.391	99.29%
q5	1.87	2.93	1.063	156.88%
q96	11.84	10.75	-1.087	90.82%
q97	2.57	2.47	-0.099	96.15%
q98	5.07	5.09	0.022	100.44%
q99	0.35	0.60	0.253	173.00%
total	1183.98	1179.13	-4.852	99.59%

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_master_03_18_2025_time.csv	log/native_master_03_17_2025_dc6e712845_time.csv	difference	percentage
q1	25.17	25.11	-0.052	99.79%
q2	26.63	28.05	1.428	105.36%
q3	33.14	32.68	-0.458	98.62%
q4	28.18	26.07	-2.112	92.50%
q5	60.65	60.22	-0.430	99.29%
q6	9.02	11.02	1.999	122.15%
q7	40.61	37.57	-3.041	92.51%
q8	63.58	63.42	-0.153	99.76%
q9	101.49	97.41	-4.083	95.98%
q10	45.49	42.30	-3.190	92.99%
q11	17.08	16.72	-0.363	97.87%
q12	17.27	17.01	-0.259	98.50%
q13	24.81	24.89	0.075	100.30%
q14	12.59	11.23	-1.356	89.23%
q15	27.27	27.23	-0.035	99.87%
q16	13.12	13.11	-0.009	99.93%
q17	73.84	74.14	0.302	100.41%
q18	112.37	112.88	0.509	100.45%
q19	16.89	17.24	0.346	102.05%
q20	22.47	25.18	2.711	112.07%
q21	176.01	171.82	-4.196	97.62%
q22	13.29	10.66	-2.634	80.18%
total	960.98	945.98	-15.000	98.44%