Skip to content

branch-3.0: [fix](load) Fix the issue of insert into select concurrent reporting causing no error url #49669 #49687

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Mar 31, 2025
Merged

branch-3.0: [fix](load) Fix the issue of insert into select concurrent reporting causing no error url #49669 #49687

dataroaring merged 1 commit into from
Mar 31, 2025

Conversation

@liaoxin01
Copy link
Contributor

@liaoxin01 liaoxin01 commented Mar 31, 2025

cherry pick from #49669

@liaoxin01
[fix](load) Fix the issue of insert into select concurrent reporting …
00f0c47 
…causing no error url (apache#49669)

Fix the issue where concurrent fragment status reporting leads to
missing error URLs. For example, when there are two fragments (one scan
fragment and one sink fragment), if the sink fragment is canceled due to
data quality issues, the scan fragment will also be canceled. However,
if the scan fragment reports its status to the Frontend (FE) first but
lacks an error URL, the INSERT INTO SELECT operation ultimately fails to
return an error URL to the user.
@liaoxin01 liaoxin01 requested a review from dataroaring as a code owner March 31, 2025 06:56
@liaoxin01
Copy link
Contributor Author

liaoxin01 commented Mar 31, 2025

run buildall

@Thearas
Copy link
Contributor

Thearas commented Mar 31, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

doris-robot commented Mar 31, 2025

TPC-H: Total hot run time: 40174 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 00f0c47297f87d1fd98d0ad1c9bf20defc5552a2, data reload: false

------ Round 1 ----------------------------------
q1	17597	7022	6646	6646
q2	2073	176	174	174
q3	10598	1072	1184	1072
q4	10554	796	696	696
q5	7748	2872	2799	2799
q6	218	133	132	132
q7	980	629	610	610
q8	9359	1997	2066	1997
q9	6672	6361	6408	6361
q10	7017	2260	2260	2260
q11	475	270	279	270
q12	397	207	204	204
q13	17780	2946	2999	2946
q14	236	198	218	198
q15	505	469	473	469
q16	689	601	595	595
q17	978	621	554	554
q18	7190	6642	6777	6642
q19	1399	1032	1041	1032
q20	474	206	209	206
q21	4079	3309	3333	3309
q22	1114	1002	1009	1002
Total cold run time: 108132 ms
Total hot run time: 40174 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6564	6627	6581	6581
q2	331	237	231	231
q3	2888	2752	2954	2752
q4	2129	1860	1830	1830
q5	5789	5780	5740	5740
q6	208	132	128	128
q7	2207	1838	1795	1795
q8	3424	3543	3575	3543
q9	8766	8925	8855	8855
q10	3625	3549	3602	3549
q11	599	484	481	481
q12	780	575	614	575
q13	8197	3160	3140	3140
q14	312	268	265	265
q15	521	493	463	463
q16	689	647	643	643
q17	1838	1623	1621	1621
q18	8264	7620	7755	7620
q19	1666	1595	1513	1513
q20	2092	1876	1860	1860
q21	5579	5312	5365	5312
q22	1143	1068	1050	1050
Total cold run time: 67611 ms
Total hot run time: 59547 ms

@doris-robot
Copy link

doris-robot commented Mar 31, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/44) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 39.05% (10227/26188)
Line Coverage 30.42% (87244/286755)
Region Coverage 29.49% (44877/152185)
Branch Coverage 26.03% (22858/87828)

@doris-robot
Copy link

doris-robot commented Mar 31, 2025

TPC-DS: Total hot run time: 197530 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 00f0c47297f87d1fd98d0ad1c9bf20defc5552a2, data reload: false

query1	1345	937	921	921
query2	6244	2067	1986	1986
query3	10941	4352	4394	4352
query4	65800	29136	23262	23262
query5	4947	444	460	444
query6	416	175	176	175
query7	5589	305	307	305
query8	297	215	206	206
query9	8899	2604	2596	2596
query10	457	269	256	256
query11	17560	15118	15778	15118
query12	163	106	107	106
query13	1507	456	450	450
query14	10379	7075	6629	6629
query15	208	183	179	179
query16	7061	490	483	483
query17	1078	559	571	559
query18	1746	308	319	308
query19	203	156	157	156
query20	119	113	111	111
query21	209	101	107	101
query22	4594	4518	4493	4493
query23	34384	34633	34563	34563
query24	6124	2964	2951	2951
query25	557	436	439	436
query26	674	169	174	169
query27	1990	372	377	372
query28	5201	2488	2455	2455
query29	736	475	489	475
query30	243	170	163	163
query31	1029	848	834	834
query32	74	57	60	57
query33	477	312	317	312
query34	930	521	531	521
query35	860	753	766	753
query36	1112	996	1039	996
query37	134	75	77	75
query38	4261	4297	4104	4104
query39	1560	1515	1512	1512
query40	219	113	111	111
query41	53	53	52	52
query42	115	109	112	109
query43	536	519	510	510
query44	1195	885	853	853
query45	187	174	179	174
query46	1164	753	740	740
query47	2110	2030	2078	2030
query48	508	381	394	381
query49	1047	407	449	407
query50	869	502	436	436
query51	7472	7418	7136	7136
query52	111	94	97	94
query53	272	191	190	190
query54	580	472	467	467
query55	78	80	80	80
query56	256	237	235	235
query57	1254	1113	1115	1113
query58	211	199	198	198
query59	3083	3052	2848	2848
query60	294	257	252	252
query61	120	112	106	106
query62	803	709	657	657
query63	212	204	198	198
query64	2061	672	677	672
query65	3271	3161	3167	3161
query66	768	293	314	293
query67	16027	15672	15710	15672
query68	3916	598	592	592
query69	450	266	257	257
query70	1170	1092	1107	1092
query71	362	251	266	251
query72	6348	4035	4006	4006
query73	750	350	351	350
query74	10556	9029	9085	9029
query75	3382	2641	2616	2616
query76	2197	1006	997	997
query77	496	268	268	268
query78	10737	9551	9494	9494
query79	1121	585	581	581
query80	842	417	428	417
query81	529	241	237	237
query82	182	95	86	86
query83	168	145	142	142
query84	284	84	82	82
query85	927	310	288	288
query86	360	294	304	294
query87	4619	4281	4293	4281
query88	3896	2425	2339	2339
query89	413	292	287	287
query90	2122	188	185	185
query91	181	148	146	146
query92	58	53	51	51
query93	1657	558	565	558
query94	832	285	299	285
query95	352	261	264	261
query96	611	275	276	275
query97	3389	3192	3165	3165
query98	222	204	198	198
query99	1558	1301	1314	1301
Total cold run time: 319671 ms
Total hot run time: 197530 ms

@doris-robot
Copy link

doris-robot commented Mar 31, 2025

ClickBench: Total hot run time: 32.8 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 00f0c47297f87d1fd98d0ad1c9bf20defc5552a2, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.07
query4	1.63	0.11	0.11
query5	0.52	0.50	0.52
query6	1.13	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.56	0.55	0.55
query11	0.14	0.10	0.10
query12	0.16	0.12	0.11
query13	0.62	0.59	0.59
query14	2.73	2.86	2.75
query15	0.88	0.82	0.82
query16	0.37	0.38	0.39
query17	1.01	1.08	1.04
query18	0.24	0.22	0.22
query19	1.91	1.79	2.08
query20	0.01	0.01	0.01
query21	15.37	0.56	0.54
query22	2.31	2.57	2.76
query23	17.28	0.94	0.69
query24	3.01	0.97	1.01
query25	0.13	0.11	0.05
query26	0.65	0.13	0.14
query27	0.06	0.05	0.04
query28	10.61	0.52	0.45
query29	12.55	3.28	3.27
query30	0.25	0.06	0.06
query31	2.85	0.39	0.38
query32	3.25	0.47	0.45
query33	2.96	2.99	3.04
query34	17.24	4.52	4.54
query35	4.55	4.58	4.57
query36	0.66	0.49	0.48
query37	0.09	0.07	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 107.08 s
Total hot run time: 32.8 s

dataroaring
dataroaring approved these changes Mar 31, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 2acf093 into apache:branch-3.0 Mar 31, 2025
21 of 24 checks passed
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@liaoxin01 @Thearas @doris-robot @dataroaring