Skip to content

Conversation

@BiteTheDDDDt
Copy link
Contributor

… repeat exprs

Proposed changes

fix core dump coz output slot'order on repeat node not match with pre repeat exprs

1# 0x00007F8A1D244B50 in /lib64/libc.so.6
 2# gsignal in /lib64/libc.so.6
 3# __GI_abort in /lib64/libc.so.6
 4# 0x000055B7CCAC16FD in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be
 5# 0x000055B7CCAB3D9A in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be
 6# google::LogMessage::SendToLog() in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be
 7# google::LogMessage::Flush() in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be
 8# google::LogMessageFatal::~LogMessageFatal() in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be
 9# doris::vectorized::ColumnVector<unsigned int> const& assert_cast<doris::vectorized::ColumnVector<unsigned int> const&, doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&) in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be
10# doris::vectorized::ColumnVector<unsigned int>::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /mnt/disk2/zhangsida/doris/be/src/vec/columns/column_vector.cpp:349
11# doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /mnt/disk2/zhangsida/doris/be/src/vec/columns/column_nullable.cpp:282
12# doris::pipeline::RepeatLocalState::get_repeated_block(doris::vectorized::Block*, int, doris::vectorized::Block*) at /mnt/disk2/zhangsida/doris/be/src/pipeline/exec/repeat_operator.cpp:154
13# doris::pipeline::RepeatOperatorX::pull(doris::RuntimeState*, doris::vectorized::Block*, bool*) const at /mnt/disk2/zhangsida/doris/be/src/pipeline/exec/repeat_operator.cpp:226
14# doris::pipeline::StatefulOperatorX<doris::pipeline::RepeatLocalState>::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk2/zhangsida/doris/be/src/pipeline/pipeline_x/operator.cpp:469
15# doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) in /mnt/disk2/zhangsida/doris/output/be/lib/doris_be

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@BiteTheDDDDt
Copy link
Contributor Author

run buidall

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38250 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9269da66ed152374948b30b3493c5c96cd8efee0, data reload: false

------ Round 1 ----------------------------------
q1	17722	4390	4252	4252
q2	2453	157	161	157
q3	10975	1104	1250	1104
q4	10739	816	817	816
q5	7623	3043	3037	3037
q6	207	129	127	127
q7	1047	589	595	589
q8	9566	2037	2017	2017
q9	7138	6616	6559	6559
q10	8853	3478	3605	3478
q11	438	216	224	216
q12	382	204	198	198
q13	18830	2865	2857	2857
q14	240	208	209	208
q15	508	464	449	449
q16	473	371	365	365
q17	965	566	575	566
q18	7374	6546	6434	6434
q19	1567	1458	1495	1458
q20	549	275	256	256
q21	3587	2884	2804	2804
q22	354	303	307	303
Total cold run time: 111590 ms
Total hot run time: 38250 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4093	4066	4063	4063
q2	322	236	229	229
q3	2944	2838	2844	2838
q4	1821	1582	1501	1501
q5	5226	5338	5286	5286
q6	195	117	117	117
q7	2239	1894	1855	1855
q8	3150	3338	3322	3322
q9	8583	8571	8561	8561
q10	3703	3723	3638	3638
q11	542	450	439	439
q12	705	552	567	552
q13	16891	2849	2843	2843
q14	292	247	261	247
q15	498	453	464	453
q16	470	427	424	424
q17	1752	1482	1477	1477
q18	7441	7121	7145	7121
q19	1620	1482	1530	1482
q20	1905	1742	1710	1710
q21	4781	4639	4684	4639
q22	530	448	437	437
Total cold run time: 69703 ms
Total hot run time: 53234 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 180839 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9269da66ed152374948b30b3493c5c96cd8efee0, data reload: false

query1	941	359	351	351
query2	7388	2077	1968	1968
query3	6716	216	211	211
query4	31180	20583	20760	20583
query5	4318	416	506	416
query6	263	192	176	176
query7	4618	297	293	293
query8	231	170	172	170
query9	9396	2276	2269	2269
query10	453	237	238	237
query11	17188	14149	14209	14149
query12	142	88	87	87
query13	1644	416	409	409
query14	13479	11109	10953	10953
query15	278	198	204	198
query16	8108	268	277	268
query17	1923	574	547	547
query18	2041	301	278	278
query19	205	164	167	164
query20	90	89	86	86
query21	200	134	132	132
query22	4621	4475	4448	4448
query23	31835	30834	31072	30834
query24	11689	2803	2758	2758
query25	618	343	370	343
query26	1789	149	152	149
query27	3046	347	366	347
query28	7706	1883	1886	1883
query29	1018	618	602	602
query30	303	145	144	144
query31	951	712	732	712
query32	89	55	53	53
query33	765	249	246	246
query34	1098	478	475	475
query35	826	603	580	580
query36	1002	856	896	856
query37	275	73	76	73
query38	3534	3436	3440	3436
query39	1396	1380	1376	1376
query40	291	106	109	106
query41	48	46	45	45
query42	100	92	96	92
query43	502	470	439	439
query44	1188	718	711	711
query45	271	262	258	258
query46	1077	665	662	662
query47	1668	1585	1568	1568
query48	446	342	349	342
query49	1251	326	333	326
query50	761	373	369	369
query51	6677	6562	6670	6562
query52	121	90	97	90
query53	345	280	279	279
query54	324	236	229	229
query55	83	77	81	77
query56	261	225	236	225
query57	1112	1007	1009	1007
query58	236	210	214	210
query59	2849	2569	2715	2569
query60	282	254	273	254
query61	114	112	113	112
query62	583	402	394	394
query63	306	279	285	279
query64	6494	3908	3725	3725
query65	3129	3028	3033	3028
query66	1423	389	348	348
query67	15233	14503	14646	14503
query68	5545	512	521	512
query69	557	386	377	377
query70	1219	1176	1145	1145
query71	441	281	276	276
query72	6301	2831	2591	2591
query73	720	319	310	310
query74	6783	6585	6536	6536
query75	3632	2738	2791	2738
query76	3645	864	894	864
query77	598	261	262	261
query78	10350	9592	9503	9503
query79	12590	511	511	511
query80	1957	416	385	385
query81	515	212	209	209
query82	605	203	200	200
query83	215	143	155	143
query84	284	77	78	77
query85	1139	322	308	308
query86	349	304	298	298
query87	3734	3562	3477	3477
query88	5446	2289	2298	2289
query89	495	352	356	352
query90	2131	172	172	172
query91	177	133	139	133
query92	59	48	46	46
query93	7097	507	476	476
query94	1349	177	180	177
query95	427	327	334	327
query96	607	270	278	270
query97	3076	2866	2882	2866
query98	225	210	203	203
query99	1060	746	738	738
Total cold run time: 313866 ms
Total hot run time: 180839 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 9269da66ed152374948b30b3493c5c96cd8efee0 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.5 seconds inserted 10000000 Rows, about 487K ops/s

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37478 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de1c0b90f23808d34fae1980d19cd95f7f5ca0a5, data reload: false

------ Round 1 ----------------------------------
q1	17639	4210	4094	4094
q2	2128	154	149	149
q3	10594	1140	1168	1140
q4	10231	776	768	768
q5	7473	2878	2911	2878
q6	207	125	121	121
q7	1003	600	566	566
q8	9322	1973	1947	1947
q9	7088	6443	6381	6381
q10	8384	3384	3450	3384
q11	443	231	218	218
q12	415	202	198	198
q13	17801	2851	2877	2851
q14	233	200	203	200
q15	495	456	462	456
q16	493	366	364	364
q17	944	619	563	563
q18	7072	6452	6465	6452
q19	1549	1411	1459	1411
q20	548	259	242	242
q21	3516	2893	2810	2810
q22	339	285	289	285
Total cold run time: 107917 ms
Total hot run time: 37478 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4266	4039	4067	4039
q2	323	232	233	232
q3	2897	2826	2814	2814
q4	1791	1586	1564	1564
q5	5151	5335	5261	5261
q6	194	116	118	116
q7	2226	1841	1805	1805
q8	3151	3253	3262	3253
q9	8536	8520	8525	8520
q10	3717	3646	3671	3646
q11	550	445	442	442
q12	721	552	521	521
q13	16916	2863	2849	2849
q14	269	248	263	248
q15	490	449	454	449
q16	460	411	417	411
q17	1713	1489	1458	1458
q18	7492	7096	6993	6993
q19	1602	1389	1484	1389
q20	1928	1695	1689	1689
q21	4828	4698	4575	4575
q22	528	435	446	435
Total cold run time: 69749 ms
Total hot run time: 52709 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 180661 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de1c0b90f23808d34fae1980d19cd95f7f5ca0a5, data reload: false

query1	928	366	355	355
query2	7471	1974	1995	1974
query3	6717	214	209	209
query4	30905	20423	20489	20423
query5	4307	397	384	384
query6	277	177	184	177
query7	4637	287	282	282
query8	228	162	170	162
query9	9128	2278	2274	2274
query10	552	238	243	238
query11	15443	14180	14148	14148
query12	142	91	84	84
query13	1635	402	412	402
query14	12851	11185	10991	10991
query15	298	192	192	192
query16	8241	257	255	255
query17	2103	569	540	540
query18	2107	288	286	286
query19	370	154	151	151
query20	90	85	90	85
query21	199	134	123	123
query22	4616	4481	4442	4442
query23	31701	31092	30870	30870
query24	10766	2793	2803	2793
query25	627	385	391	385
query26	1180	152	156	152
query27	2499	347	344	344
query28	6742	1893	1886	1886
query29	889	653	625	625
query30	302	151	151	151
query31	976	736	728	728
query32	94	59	55	55
query33	857	254	255	254
query34	949	473	467	467
query35	812	606	608	606
query36	990	890	857	857
query37	121	79	75	75
query38	3567	3447	3421	3421
query39	1451	1360	1352	1352
query40	209	113	109	109
query41	46	46	47	46
query42	98	93	95	93
query43	478	471	464	464
query44	1087	712	712	712
query45	270	225	241	225
query46	1077	668	664	664
query47	1648	1616	1573	1573
query48	447	357	355	355
query49	1084	334	330	330
query50	749	365	356	356
query51	6699	6586	6488	6488
query52	99	91	82	82
query53	333	266	264	264
query54	299	234	261	234
query55	83	85	77	77
query56	245	224	216	216
query57	1071	995	1023	995
query58	239	202	194	194
query59	2737	2596	2585	2585
query60	336	246	236	236
query61	94	91	92	91
query62	609	402	411	402
query63	305	270	274	270
query64	5598	3872	3816	3816
query65	3038	3003	2961	2961
query66	881	336	340	336
query67	14886	14389	14596	14389
query68	5500	508	514	508
query69	555	377	364	364
query70	1237	1148	1168	1148
query71	429	276	276	276
query72	6323	2800	2615	2615
query73	703	312	314	312
query74	6987	6536	6551	6536
query75	3393	2759	2767	2759
query76	3531	928	866	866
query77	408	271	259	259
query78	10439	9540	9688	9540
query79	8055	517	511	511
query80	2283	414	398	398
query81	578	219	211	211
query82	1343	211	205	205
query83	279	149	152	149
query84	295	84	83	83
query85	1840	322	313	313
query86	507	311	279	279
query87	3763	3527	3520	3520
query88	5152	2304	2270	2270
query89	469	358	355	355
query90	2023	169	172	169
query91	169	133	137	133
query92	66	46	48	46
query93	6038	508	479	479
query94	1215	178	173	173
query95	437	333	326	326
query96	604	268	271	268
query97	3019	2877	2878	2877
query98	223	208	211	208
query99	1136	744	782	744
Total cold run time: 301512 ms
Total hot run time: 180661 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit de1c0b90f23808d34fae1980d19cd95f7f5ca0a5 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.2 seconds inserted 10000000 Rows, about 450K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 26, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@BiteTheDDDDt BiteTheDDDDt merged commit d6ddd7a into apache:master Mar 26, 2024
yiguolei pushed a commit that referenced this pull request Mar 26, 2024
…match with pre repeat exprs (#32662)

fix core dump coz output slot'order on repeat node not match with pre repeat exprs
BiteTheDDDDt added a commit to BiteTheDDDDt/incubator-doris that referenced this pull request May 14, 2024
…match with pre repeat exprs (apache#32662)

fix core dump coz output slot'order on repeat node not match with pre repeat exprs
xiaokang pushed a commit that referenced this pull request May 20, 2024
…match with pre repeat exprs (#32662) (#34803)

fix core dump coz output slot'order on repeat node not match with pre repeat exprs
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…match with pre repeat exprs (apache#32662) (apache#34803)

fix core dump coz output slot'order on repeat node not match with pre repeat exprs
@BiteTheDDDDt BiteTheDDDDt deleted the fix_0322_3 branch January 20, 2025 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.11-merged p0_c reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants