Skip to content

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Mar 7, 2025

What problem does this PR solve?

Related PR: #44267
Problem Summary:
RecordReader will use ProcessBuilder to start a hotspot process, which may be stuck, so use another process to kill this stuck process.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32870 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 24a3c5e9c596eef1857db8acae647919e14f4824, data reload: false

------ Round 1 ----------------------------------
q1	17597	5167	5148	5148
q2	2052	285	174	174
q3	10424	1328	688	688
q4	10218	1027	532	532
q5	7524	2408	2352	2352
q6	194	170	136	136
q7	896	727	618	618
q8	9325	1308	1115	1115
q9	5166	4892	4757	4757
q10	6834	2326	1888	1888
q11	472	278	254	254
q12	344	354	228	228
q13	17776	3677	3107	3107
q14	229	234	206	206
q15	534	490	498	490
q16	630	632	585	585
q17	585	861	359	359
q18	6844	6524	6477	6477
q19	1223	957	549	549
q20	324	317	191	191
q21	2790	2234	2038	2038
q22	1063	1039	978	978
Total cold run time: 103044 ms
Total hot run time: 32870 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5187	5152	5166	5152
q2	248	324	230	230
q3	2171	2654	2291	2291
q4	1444	1837	1408	1408
q5	4282	4154	4164	4154
q6	212	165	123	123
q7	1876	1981	1811	1811
q8	2627	2734	2586	2586
q9	7236	7170	7137	7137
q10	3028	3175	2801	2801
q11	580	525	509	509
q12	668	747	611	611
q13	3442	3896	3234	3234
q14	277	298	273	273
q15	506	488	505	488
q16	647	682	637	637
q17	1127	1599	1335	1335
q18	7702	7610	7407	7407
q19	833	822	878	822
q20	1973	2026	1862	1862
q21	5595	5053	4815	4815
q22	1131	1053	1011	1011
Total cold run time: 52792 ms
Total hot run time: 50697 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185421 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 24a3c5e9c596eef1857db8acae647919e14f4824, data reload: false

query1	974	404	378	378
query2	6523	1939	1883	1883
query3	6800	222	215	215
query4	27040	23865	23194	23194
query5	4745	662	503	503
query6	326	204	188	188
query7	4611	506	291	291
query8	307	254	236	236
query9	8642	2512	2501	2501
query10	468	322	260	260
query11	15472	15081	14967	14967
query12	165	111	115	111
query13	1648	516	376	376
query14	9637	6133	6305	6133
query15	210	187	180	180
query16	7538	662	464	464
query17	1154	724	549	549
query18	1986	396	294	294
query19	189	186	152	152
query20	124	120	115	115
query21	208	123	100	100
query22	4344	4486	4259	4259
query23	34116	32936	33035	32936
query24	8147	2368	2386	2368
query25	528	455	391	391
query26	1236	276	153	153
query27	2695	500	328	328
query28	4369	2384	2367	2367
query29	736	566	453	453
query30	284	219	200	200
query31	965	865	781	781
query32	76	71	67	67
query33	579	381	317	317
query34	787	855	497	497
query35	827	845	744	744
query36	955	981	866	866
query37	118	102	78	78
query38	4162	4181	4358	4181
query39	1458	1425	1410	1410
query40	212	120	108	108
query41	61	59	57	57
query42	123	110	106	106
query43	508	531	499	499
query44	1279	795	788	788
query45	180	175	176	175
query46	837	1036	629	629
query47	1803	1854	1757	1757
query48	372	411	304	304
query49	812	556	433	433
query50	687	727	400	400
query51	4180	4212	4129	4129
query52	107	106	93	93
query53	229	261	184	184
query54	478	475	415	415
query55	83	78	85	78
query56	285	280	250	250
query57	1116	1159	1095	1095
query58	245	234	238	234
query59	2568	2781	2642	2642
query60	293	277	258	258
query61	122	125	116	116
query62	791	731	666	666
query63	233	188	198	188
query64	4336	1045	683	683
query65	4419	4325	4328	4325
query66	1082	413	292	292
query67	15798	15624	15315	15315
query68	8120	870	510	510
query69	471	308	269	269
query70	1192	1118	1110	1110
query71	458	302	269	269
query72	5642	3547	3705	3547
query73	764	722	347	347
query74	9205	8903	9013	8903
query75	3806	3189	2664	2664
query76	3683	1193	769	769
query77	792	371	295	295
query78	10141	10158	9250	9250
query79	2361	835	591	591
query80	607	525	449	449
query81	496	264	225	225
query82	556	124	96	96
query83	167	172	155	155
query84	240	98	76	76
query85	787	446	307	307
query86	381	312	284	284
query87	4480	4384	4466	4384
query88	3653	2229	2267	2229
query89	395	322	292	292
query90	1855	212	212	212
query91	136	137	111	111
query92	82	61	58	58
query93	1665	1055	589	589
query94	656	413	310	310
query95	357	267	258	258
query96	480	558	278	278
query97	3357	3448	3393	3393
query98	228	213	204	204
query99	1440	1396	1282	1282
Total cold run time: 277041 ms
Total hot run time: 185421 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 24a3c5e9c596eef1857db8acae647919e14f4824, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.04	0.03
query3	0.24	0.06	0.06
query4	1.62	0.11	0.11
query5	0.55	0.55	0.54
query6	1.20	0.72	0.71
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.53	0.51
query10	0.57	0.61	0.58
query11	0.16	0.11	0.10
query12	0.14	0.11	0.11
query13	0.62	0.62	0.61
query14	2.82	2.70	2.72
query15	0.92	0.85	0.85
query16	0.39	0.39	0.39
query17	1.03	1.05	1.01
query18	0.21	0.20	0.20
query19	1.92	1.78	1.96
query20	0.01	0.01	0.01
query21	15.36	0.92	0.54
query22	0.76	1.26	0.71
query23	14.81	1.39	0.61
query24	7.30	1.11	0.90
query25	0.50	0.29	0.16
query26	0.70	0.17	0.13
query27	0.06	0.05	0.05
query28	9.38	0.89	0.44
query29	12.53	4.00	3.34
query30	0.25	0.09	0.06
query31	2.82	0.58	0.39
query32	3.23	0.55	0.46
query33	2.98	3.08	3.11
query34	16.05	5.22	4.60
query35	4.58	4.55	4.59
query36	0.67	0.49	0.48
query37	0.09	0.06	0.06
query38	0.06	0.05	0.03
query39	0.04	0.03	0.03
query40	0.18	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.64 s
Total hot run time: 31.18 s

@suxiaogang223 suxiaogang223 deleted the kill_jol branch March 11, 2025 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants