Skip to content

[branch-2.0](cherry-pick) Fix task repeat attach task DCHECK failed #32914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 2 commits into from
Mar 27, 2024
Merged

[branch-2.0](cherry-pick) Fix task repeat attach task DCHECK failed #32914

yiguolei merged 2 commits into from
Mar 27, 2024

Conversation

@xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Mar 27, 2024

Proposed changes

pick #32784

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Mar 27, 2024

run buildall

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Mar 27, 2024

run buildall

github-actions[bot]
github-actions bot reviewed Mar 27, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TPC-H: Total hot run time: 50018 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4271e382dbe3b0ab92b4b42381ca0be922995559, data reload: false

------ Round 1 ----------------------------------
q1	17798	4421	4431	4421
q2	2075	153	155	153
q3	10488	1913	1950	1913
q4	10338	1258	1335	1258
q5	8549	3964	3992	3964
q6	231	124	126	124
q7	2046	1573	1606	1573
q8	9250	2732	2732	2732
q9	10830	10604	10349	10349
q10	8671	3501	3504	3501
q11	426	244	242	242
q12	461	300	298	298
q13	18348	3935	4021	3935
q14	363	322	338	322
q15	499	440	462	440
q16	690	607	587	587
q17	1127	971	975	971
q18	7195	6945	7077	6945
q19	1661	1566	1549	1549
q20	517	290	288	288
q21	4468	4132	4063	4063
q22	518	418	390	390
Total cold run time: 116549 ms
Total hot run time: 50018 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4338	4293	4293
q2	315	218	235	218
q3	4181	4130	4175	4130
q4	2738	2758	2764	2758
q5	7248	7239	7210	7210
q6	236	119	119	119
q7	3265	2818	2815	2815
q8	4358	4457	4498	4457
q9	17134	17022	16893	16893
q10	4256	4297	4276	4276
q11	760	669	735	669
q12	1014	848	852	848
q13	7287	3748	3738	3738
q14	449	430	428	428
q15	496	452	444	444
q16	761	706	736	706
q17	3939	3840	3844	3840
q18	8797	8851	8911	8851
q19	1707	1720	1692	1692
q20	2358	2129	2129	2129
q21	8551	8613	8577	8577
q22	1058	938	957	938
Total cold run time: 85236 ms
Total hot run time: 80029 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TeamCity be ut coverage result:
Function Coverage: 37.80% (8046/21286)
Line Coverage: 29.47% (65718/223017)
Region Coverage: 28.93% (33816/116904)
Branch Coverage: 24.78% (17362/70066)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4271e382dbe3b0ab92b4b42381ca0be922995559_4271e382dbe3b0ab92b4b42381ca0be922995559/report/index.html

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TPC-DS: Total hot run time: 201208 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4271e382dbe3b0ab92b4b42381ca0be922995559, data reload: false

query1	928	397	377	377
query2	6538	2292	2084	2084
query3	6925	197	199	197
query4	19917	18023	17936	17936
query5	19716	6548	6500	6500
query6	322	210	218	210
query7	4393	301	294	294
query8	266	257	242	242
query9	3161	2716	2637	2637
query10	414	293	307	293
query11	11318	10772	10688	10688
query12	125	74	73	73
query13	5567	619	612	612
query14	18639	13800	13378	13378
query15	378	244	242	242
query16	6436	268	254	254
query17	1685	1455	859	859
query18	2322	398	403	398
query19	207	141	146	141
query20	76	78	76	76
query21	184	90	95	90
query22	5301	5078	5070	5070
query23	32597	32015	32092	32015
query24	6734	6512	6480	6480
query25	520	432	425	425
query26	520	157	152	152
query27	1876	292	299	292
query28	6119	2271	2243	2243
query29	2903	2580	2716	2580
query30	242	161	159	159
query31	917	721	730	721
query32	67	58	59	58
query33	394	257	247	247
query34	855	471	486	471
query35	1115	926	952	926
query36	1247	1094	1185	1094
query37	91	64	59	59
query38	3052	2915	2950	2915
query39	1343	1313	1305	1305
query40	200	93	93	93
query41	36	32	31	31
query42	79	85	78	78
query43	715	614	607	607
query44	1167	730	722	722
query45	241	224	229	224
query46	1223	958	989	958
query47	1890	1751	1786	1751
query48	969	680	660	660
query49	608	372	358	358
query50	868	593	585	585
query51	4721	4707	4714	4707
query52	84	79	75	75
query53	432	317	311	311
query54	2628	2491	2468	2468
query55	89	80	80	80
query56	218	211	183	183
query57	1145	1190	1094	1094
query58	211	193	198	193
query59	3356	3364	3362	3362
query60	212	201	194	194
query61	82	79	84	79
query62	871	482	506	482
query63	468	334	333	333
query64	2522	1488	1385	1385
query65	3623	3550	3539	3539
query66	769	377	360	360
query67	17140	16928	15170	15170
query68	8711	637	652	637
query69	595	338	344	338
query70	1602	1302	1417	1302
query71	415	318	298	298
query72	6539	3388	3402	3388
query73	730	323	326	323
query74	6227	5895	5841	5841
query75	5145	3632	3722	3632
query76	5159	1141	1232	1141
query77	846	251	246	246
query78	12483	11719	11758	11719
query79	7182	610	643	610
query80	1001	402	378	378
query81	482	227	229	227
query82	1547	106	93	93
query83	162	134	128	128
query84	256	68	66	66
query85	848	285	275	275
query86	334	293	338	293
query87	3250	3019	2986	2986
query88	4800	2308	2310	2308
query89	362	272	270	270
query90	1931	209	198	198
query91	148	114	115	114
query92	57	50	51	50
query93	3632	546	606	546
query94	713	203	206	203
query95	1109	1071	1052	1052
query96	634	325	321	321
query97	6459	6275	6396	6275
query98	179	172	172	172
query99	2958	903	920	903
Total cold run time: 308473 ms
Total hot run time: 201208 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

ClickBench: Total hot run time: 30.65 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4271e382dbe3b0ab92b4b42381ca0be922995559, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.04	0.05
query4	1.82	0.08	0.08
query5	0.52	0.52	0.51
query6	1.26	0.60	0.61
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.52	0.49	0.47
query10	0.55	0.53	0.53
query11	0.12	0.09	0.08
query12	0.11	0.09	0.09
query13	0.62	0.61	0.62
query14	0.78	0.79	0.79
query15	0.78	0.76	0.75
query16	0.38	0.37	0.35
query17	1.01	1.01	1.02
query18	0.21	0.26	0.26
query19	1.83	1.78	1.87
query20	0.02	0.01	0.02
query21	15.47	0.56	0.54
query22	2.24	2.62	1.60
query23	17.24	1.01	0.90
query24	4.66	3.09	1.11
query25	0.39	0.11	0.05
query26	0.72	0.15	0.18
query27	0.05	0.04	0.03
query28	6.32	0.82	0.73
query29	12.63	2.31	2.26
query30	0.60	0.54	0.49
query31	2.80	0.40	0.38
query32	3.36	0.50	0.50
query33	3.07	3.05	3.05
query34	15.23	4.82	4.77
query35	4.87	4.84	4.83
query36	1.04	1.00	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.03
query39	0.02	0.01	0.01
query40	0.16	0.15	0.13
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.03	0.01	0.02
Total cold run time: 101.99 s
Total hot run time: 30.65 s

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4271e382dbe3b0ab92b4b42381ca0be922995559 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

@yiguolei yiguolei merged commit 2dca398 into apache:branch-2.0 Mar 27, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 27, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Mar 27, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 27, 2024

PR approved by anyone and no changes requested.

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@xinyiZzz
[branch-2.0](cherry-pick) Fix task repeat attach task DCHECK failed (a…
21634c8 
…pache#32914)

* 1

* 2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@xinyiZzz @doris-robot @yiguolei