Skip to content

Conversation

@zhangstar333
Copy link
Contributor

cherry-pick from master #33295

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

… check type (apache#33295)

* [Bug](array) fix array column core dump in get_shrinked_column as not check type

* add function could_shrinked_column
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 50292 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 04a6783c1e0449031e3b06fd6def42c05db0b407, data reload: false

------ Round 1 ----------------------------------
q1	17553	4396	4332	4332
q2	2040	150	142	142
q3	10468	1904	2016	1904
q4	10354	1259	1338	1259
q5	8546	4000	3993	3993
q6	233	122	121	121
q7	2052	1620	1620	1620
q8	9293	2715	2733	2715
q9	11087	11453	10518	10518
q10	8646	3527	3540	3527
q11	441	240	259	240
q12	473	295	301	295
q13	18341	4022	4054	4022
q14	350	346	323	323
q15	514	464	456	456
q16	718	592	613	592
q17	1136	972	978	972
q18	7311	6857	7045	6857
q19	1679	1590	1566	1566
q20	521	303	318	303
q21	4473	4158	4144	4144
q22	521	391	401	391
Total cold run time: 116750 ms
Total hot run time: 50292 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4333	4321	4308	4308
q2	322	219	224	219
q3	4163	4160	4165	4160
q4	2769	2758	2762	2758
q5	7310	7184	7257	7184
q6	237	118	121	118
q7	3290	2841	2847	2841
q8	4375	4543	4510	4510
q9	17189	16942	16983	16942
q10	4222	4273	4295	4273
q11	740	682	701	682
q12	1019	858	869	858
q13	6441	3752	3755	3752
q14	457	421	425	421
q15	488	458	457	457
q16	758	694	706	694
q17	3846	3817	3859	3817
q18	8901	8840	8844	8840
q19	1697	1706	1644	1644
q20	2392	2149	2147	2147
q21	8610	8548	8580	8548
q22	1067	958	934	934
Total cold run time: 84626 ms
Total hot run time: 80107 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 200666 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 04a6783c1e0449031e3b06fd6def42c05db0b407, data reload: false

query1	917	406	384	384
query2	6539	2211	2058	2058
query3	6918	202	198	198
query4	20015	17947	17837	17837
query5	19728	6490	6473	6473
query6	285	213	218	213
query7	4161	299	289	289
query8	249	229	270	229
query9	3213	2769	2702	2702
query10	409	299	309	299
query11	11363	10628	10612	10612
query12	117	76	72	72
query13	5582	641	629	629
query14	17614	13155	13544	13155
query15	397	225	233	225
query16	6462	268	250	250
query17	1728	1468	891	891
query18	2296	410	402	402
query19	206	143	147	143
query20	77	75	76	75
query21	188	97	91	91
query22	5223	4914	5055	4914
query23	32680	31825	32126	31825
query24	6923	6473	6584	6473
query25	535	431	426	426
query26	617	166	157	157
query27	2034	289	291	289
query28	6115	2271	2230	2230
query29	2820	2850	2833	2833
query30	236	161	165	161
query31	905	702	721	702
query32	69	62	62	62
query33	386	260	243	243
query34	838	460	476	460
query35	1102	902	923	902
query36	1311	1228	1117	1117
query37	88	62	58	58
query38	3106	2933	2890	2890
query39	1376	1304	1320	1304
query40	245	92	94	92
query41	35	32	33	32
query42	87	84	84	84
query43	594	625	593	593
query44	1106	719	719	719
query45	238	227	224	224
query46	1228	954	958	954
query47	1841	1592	1783	1592
query48	982	693	650	650
query49	620	365	363	363
query50	874	595	615	595
query51	4751	4604	4660	4604
query52	93	79	74	74
query53	434	315	315	315
query54	2641	2474	2437	2437
query55	85	75	79	75
query56	219	195	208	195
query57	1159	1053	1066	1053
query58	209	204	211	204
query59	3620	3439	3246	3246
query60	206	187	196	187
query61	82	82	85	82
query62	791	451	518	451
query63	472	334	339	334
query64	2475	1477	1423	1423
query65	3625	3552	3533	3533
query66	787	374	377	374
query67	15536	15908	15144	15144
query68	10034	667	663	663
query69	569	349	346	346
query70	1791	1315	1313	1313
query71	403	295	307	295
query72	6542	3429	3450	3429
query73	734	317	316	316
query74	6275	5860	5788	5788
query75	5348	3709	3722	3709
query76	6435	1137	1203	1137
query77	1081	253	243	243
query78	12702	12006	11826	11826
query79	7526	646	633	633
query80	995	400	387	387
query81	492	230	234	230
query82	1565	105	99	99
query83	172	135	131	131
query84	260	69	66	66
query85	859	277	290	277
query86	326	292	299	292
query87	3231	3000	3038	3000
query88	4642	2322	2309	2309
query89	374	285	335	285
query90	1915	209	210	209
query91	150	114	117	114
query92	55	53	55	53
query93	5850	579	606	579
query94	643	203	204	203
query95	1109	1073	1057	1057
query96	628	323	324	323
query97	6498	6395	6391	6391
query98	190	179	169	169
query99	2841	892	1049	892
Total cold run time: 312098 ms
Total hot run time: 200666 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.83% (8064/21315)
Line Coverage: 29.52% (65887/223183)
Region Coverage: 28.99% (33903/116951)
Branch Coverage: 24.85% (17413/70072)
Coverage Report: http://coverage.selectdb-in.cc/coverage/04a6783c1e0449031e3b06fd6def42c05db0b407_04a6783c1e0449031e3b06fd6def42c05db0b407/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 30.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04a6783c1e0449031e3b06fd6def42c05db0b407, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.04
query4	1.78	0.07	0.06
query5	0.53	0.53	0.52
query6	1.23	0.62	0.63
query7	0.02	0.02	0.02
query8	0.04	0.02	0.02
query9	0.53	0.48	0.50
query10	0.53	0.54	0.54
query11	0.12	0.08	0.08
query12	0.11	0.10	0.09
query13	0.63	0.61	0.61
query14	0.79	0.77	0.78
query15	0.78	0.76	0.75
query16	0.37	0.37	0.36
query17	1.01	1.01	1.00
query18	0.21	0.28	0.25
query19	1.92	1.89	1.87
query20	0.01	0.00	0.01
query21	15.46	0.57	0.55
query22	2.06	1.90	1.44
query23	17.17	1.06	0.99
query24	5.12	1.04	1.42
query25	0.33	0.09	0.06
query26	0.63	0.16	0.14
query27	0.05	0.04	0.03
query28	7.57	0.75	0.75
query29	12.62	2.33	2.22
query30	0.61	0.52	0.53
query31	2.83	0.39	0.37
query32	3.37	0.49	0.49
query33	3.04	3.11	3.05
query34	15.27	4.79	4.80
query35	4.88	4.83	4.83
query36	1.06	1.01	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.13
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 103.4 s
Total hot run time: 30.64 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 04a6783c1e0449031e3b06fd6def42c05db0b407 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.3 seconds inserted 10000000 Rows, about 492K ops/s

@yiguolei yiguolei merged commit 242e299 into apache:branch-2.0 Apr 11, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
… check type (apache#33295) (apache#33552)

* [Bug](array) fix array column core dump in get_shrinked_column as not check type

* add function could_shrinked_column
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants