Skip to content

Conversation

@amorynan
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@amorynan
Copy link
Contributor Author

run buildall

@xiaokang xiaokang changed the title [improve](meta) improve upgrade action for map column from 1.2.* to 2.0.* [fix](map) fix core dump when upgrading from 1.2.x to 2.0.x with map datatype column Jun 20, 2024
@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49674 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit be03238ab1477e7c3078f62e6d516185560d1289, data reload: false

------ Round 1 ----------------------------------
q1	18141	4347	4333	4333
q2	2071	161	155	155
q3	10304	1870	1917	1870
q4	10352	1240	1307	1240
q5	8474	3866	3890	3866
q6	243	125	146	125
q7	2054	1604	1566	1566
q8	9501	2692	2698	2692
q9	13669	10216	10144	10144
q10	8688	3559	3529	3529
q11	410	250	261	250
q12	470	305	307	305
q13	18338	3956	4033	3956
q14	362	336	334	334
q15	510	453	456	453
q16	664	569	576	569
q17	1129	913	899	899
q18	7292	7002	6891	6891
q19	1824	1620	1635	1620
q20	556	322	307	307
q21	4422	4173	4120	4120
q22	541	450	459	450
Total cold run time: 120015 ms
Total hot run time: 49674 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4307	4294	4336	4294
q2	321	226	226	226
q3	4150	4158	4134	4134
q4	2759	2736	2744	2736
q5	7222	7139	7148	7139
q6	238	121	120	120
q7	3265	2809	2845	2809
q8	4387	4447	4489	4447
q9	16869	16758	16714	16714
q10	4272	4256	4241	4241
q11	758	705	706	705
q12	1027	863	857	857
q13	7011	3728	3762	3728
q14	468	421	433	421
q15	501	466	447	447
q16	734	675	701	675
q17	3806	3905	3821	3821
q18	8870	8685	8824	8685
q19	1763	1665	1674	1665
q20	2408	2141	2110	2110
q21	8470	8489	8522	8489
q22	1044	987	1025	987
Total cold run time: 84650 ms
Total hot run time: 79450 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.86% (8107/21415)
Line Coverage: 29.53% (66349/224718)
Region Coverage: 28.99% (34188/117919)
Branch Coverage: 24.86% (17553/70602)
Coverage Report: http://coverage.selectdb-in.cc/coverage/be03238ab1477e7c3078f62e6d516185560d1289_be03238ab1477e7c3078f62e6d516185560d1289/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 203836 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit be03238ab1477e7c3078f62e6d516185560d1289, data reload: false

query1	930	418	372	372
query2	6520	2884	2695	2695
query3	6920	210	205	205
query4	21097	17854	17988	17854
query5	19741	6514	6472	6472
query6	288	222	236	222
query7	4160	306	304	304
query8	445	436	472	436
query9	3145	2685	2607	2607
query10	404	288	297	288
query11	11404	10654	10649	10649
query12	124	77	76	76
query13	5597	692	687	687
query14	17943	13527	14166	13527
query15	365	256	251	251
query16	6446	293	268	268
query17	1724	1457	877	877
query18	2598	421	415	415
query19	205	153	152	152
query20	82	75	80	75
query21	187	95	95	95
query22	5298	5083	4995	4995
query23	32550	31919	32023	31919
query24	6908	6503	6488	6488
query25	518	439	419	419
query26	530	159	159	159
query27	1895	302	299	299
query28	6149	2359	2287	2287
query29	3026	2696	2782	2696
query30	240	166	168	166
query31	925	729	764	729
query32	72	63	67	63
query33	410	256	248	248
query34	843	471	469	469
query35	1124	891	901	891
query36	1303	1308	1255	1255
query37	89	59	62	59
query38	3105	2899	2900	2899
query39	1381	1325	1337	1325
query40	216	95	101	95
query41	46	43	46	43
query42	87	85	78	78
query43	764	696	723	696
query44	1159	728	724	724
query45	248	240	235	235
query46	1226	978	950	950
query47	1838	1827	1638	1638
query48	1021	713	713	713
query49	622	374	384	374
query50	858	656	601	601
query51	4802	4622	4737	4622
query52	99	84	92	84
query53	459	333	330	330
query54	2652	2367	2461	2367
query55	98	86	83	83
query56	226	227	220	220
query57	1145	1009	1065	1009
query58	211	210	202	202
query59	4382	4175	3993	3993
query60	216	228	226	226
query61	102	96	102	96
query62	801	513	497	497
query63	487	350	343	343
query64	2590	1572	1436	1436
query65	3667	3555	3560	3555
query66	811	395	381	381
query67	15942	15192	16723	15192
query68	9205	661	653	653
query69	593	358	369	358
query70	1456	1462	1411	1411
query71	415	316	325	316
query72	6541	3489	3464	3464
query73	741	317	317	317
query74	6342	5851	5878	5851
query75	5357	3794	3745	3745
query76	5737	1152	1176	1152
query77	931	256	260	256
query78	12477	12044	12364	12044
query79	9826	614	639	614
query80	1099	413	400	400
query81	493	234	238	234
query82	782	99	98	98
query83	174	135	129	129
query84	262	69	69	69
query85	854	320	318	318
query86	336	290	293	290
query87	3238	3022	3040	3022
query88	4448	2307	2294	2294
query89	383	291	288	288
query90	1974	210	217	210
query91	176	142	156	142
query92	61	54	53	53
query93	4573	573	572	572
query94	743	219	215	215
query95	1134	1059	1069	1059
query96	644	319	317	317
query97	6455	6432	6260	6260
query98	191	177	172	172
query99	2943	837	872	837
Total cold run time: 314050 ms
Total hot run time: 203836 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit be03238ab1477e7c3078f62e6d516185560d1289, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.04	0.05
query4	1.79	0.07	0.06
query5	0.54	0.53	0.51
query6	1.29	0.61	0.59
query7	0.01	0.02	0.01
query8	0.03	0.03	0.02
query9	0.52	0.49	0.48
query10	0.54	0.53	0.53
query11	0.12	0.08	0.08
query12	0.13	0.09	0.09
query13	0.61	0.61	0.62
query14	0.79	0.78	0.78
query15	0.78	0.77	0.76
query16	0.36	0.37	0.37
query17	0.96	1.00	1.02
query18	0.22	0.24	0.25
query19	1.93	1.80	1.83
query20	0.01	0.01	0.01
query21	15.46	0.58	0.55
query22	2.25	2.11	1.54
query23	17.16	1.06	0.92
query24	5.07	0.78	1.21
query25	0.36	0.11	0.07
query26	0.53	0.16	0.16
query27	0.05	0.04	0.04
query28	8.07	0.78	0.72
query29	12.66	2.35	2.22
query30	0.54	0.57	0.53
query31	2.80	0.39	0.38
query32	3.37	0.49	0.49
query33	3.07	3.10	3.04
query34	15.25	4.80	4.79
query35	4.86	4.85	4.83
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.03
query39	0.02	0.02	0.01
query40	0.16	0.15	0.15
query41	0.08	0.01	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 103.92 s
Total hot run time: 30.37 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit be03238ab1477e7c3078f62e6d516185560d1289 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.86% (8107/21415)
Line Coverage: 29.52% (66343/224717)
Region Coverage: 28.99% (34190/117926)
Branch Coverage: 24.86% (17552/70602)
Coverage Report: http://coverage.selectdb-in.cc/coverage/218339d097d18d369bd0bc97bcf4e2fc6c9919c0_218339d097d18d369bd0bc97bcf4e2fc6c9919c0/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49496 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 218339d097d18d369bd0bc97bcf4e2fc6c9919c0, data reload: false

------ Round 1 ----------------------------------
q1	17633	4393	4360	4360
q2	2101	153	148	148
q3	10445	1883	1888	1883
q4	10402	1232	1315	1232
q5	8732	3880	3892	3880
q6	238	126	121	121
q7	2066	1578	1605	1578
q8	9278	2720	2706	2706
q9	10644	10255	10138	10138
q10	8624	3513	3491	3491
q11	424	245	248	245
q12	469	298	298	298
q13	18364	3931	3998	3931
q14	358	319	333	319
q15	508	448	465	448
q16	688	564	573	564
q17	1151	989	986	986
q18	7222	6771	6908	6771
q19	1788	1649	1574	1574
q20	517	308	321	308
q21	4430	4072	4073	4072
q22	527	443	461	443
Total cold run time: 116609 ms
Total hot run time: 49496 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4349	4319	4284	4284
q2	320	224	227	224
q3	4142	4143	4152	4143
q4	2735	2741	2740	2740
q5	7091	7045	7103	7045
q6	238	122	118	118
q7	3275	2814	2869	2814
q8	4356	4448	4467	4448
q9	16927	16689	16845	16689
q10	4274	4298	4229	4229
q11	745	707	691	691
q12	1024	852	860	852
q13	7412	3739	3712	3712
q14	459	423	432	423
q15	505	462	457	457
q16	732	691	679	679
q17	3881	3934	3837	3837
q18	8820	8743	8735	8735
q19	1775	1730	1640	1640
q20	2364	2151	2126	2126
q21	8608	8426	8594	8426
q22	1056	983	982	982
Total cold run time: 85088 ms
Total hot run time: 79294 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204138 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 218339d097d18d369bd0bc97bcf4e2fc6c9919c0, data reload: false

query1	919	421	377	377
query2	6524	2783	2676	2676
query3	6928	206	204	204
query4	21456	18126	18017	18017
query5	19725	6505	6518	6505
query6	278	219	230	219
query7	4167	315	304	304
query8	416	467	413	413
query9	3042	2640	2557	2557
query10	412	309	305	305
query11	11422	10632	10601	10601
query12	124	78	79	78
query13	5601	684	685	684
query14	17765	13168	13744	13168
query15	363	245	241	241
query16	6476	287	257	257
query17	1702	1434	865	865
query18	2339	413	403	403
query19	210	149	152	149
query20	87	78	82	78
query21	197	96	97	96
query22	5293	4997	5072	4997
query23	32695	31960	32094	31960
query24	7103	6501	6563	6501
query25	510	440	411	411
query26	602	165	160	160
query27	2003	292	296	292
query28	6188	2329	2291	2291
query29	3009	2822	2701	2701
query30	251	166	166	166
query31	905	737	782	737
query32	71	64	58	58
query33	400	258	249	249
query34	848	464	475	464
query35	1124	906	925	906
query36	1216	1260	1121	1121
query37	90	58	62	58
query38	3068	2969	2941	2941
query39	1386	1332	1316	1316
query40	206	95	92	92
query41	45	44	43	43
query42	81	79	79	79
query43	768	676	733	676
query44	1152	725	718	718
query45	241	239	238	238
query46	1224	963	980	963
query47	1892	1860	1941	1860
query48	1017	709	704	704
query49	635	360	355	355
query50	866	605	630	605
query51	4719	4601	4791	4601
query52	91	92	81	81
query53	465	318	317	317
query54	2643	2460	2480	2460
query55	85	75	89	75
query56	237	215	205	205
query57	1163	1091	1023	1023
query58	229	199	188	188
query59	4087	4155	4189	4155
query60	204	208	194	194
query61	95	95	96	95
query62	775	486	439	439
query63	483	333	340	333
query64	2591	1555	1507	1507
query65	3629	3552	3570	3552
query66	791	376	375	375
query67	17387	16850	16032	16032
query68	5931	650	639	639
query69	568	350	360	350
query70	1504	1505	1377	1377
query71	404	303	312	303
query72	6405	3450	3511	3450
query73	731	315	310	310
query74	6311	5830	5855	5830
query75	4588	3591	3737	3591
query76	3645	1145	1213	1145
query77	576	248	252	248
query78	12392	11661	11707	11661
query79	9036	649	653	649
query80	3502	396	415	396
query81	519	234	234	234
query82	1750	96	100	96
query83	197	139	133	133
query84	266	72	71	71
query85	1382	320	312	312
query86	361	321	319	319
query87	3202	3048	2979	2979
query88	5294	2288	2311	2288
query89	394	321	325	321
query90	1747	204	214	204
query91	173	141	142	141
query92	57	56	54	54
query93	5862	565	584	565
query94	887	215	210	210
query95	1118	1059	1041	1041
query96	649	324	336	324
query97	6478	6350	6340	6340
query98	201	180	175	175
query99	2995	891	1049	891
Total cold run time: 313841 ms
Total hot run time: 204138 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.35 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 218339d097d18d369bd0bc97bcf4e2fc6c9919c0, data reload: false

query1	0.02	0.01	0.02
query2	0.07	0.03	0.02
query3	0.24	0.04	0.05
query4	1.79	0.06	0.05
query5	0.53	0.53	0.52
query6	1.30	0.61	0.61
query7	0.01	0.01	0.02
query8	0.03	0.02	0.02
query9	0.53	0.48	0.47
query10	0.54	0.54	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.63	0.62	0.61
query14	0.79	0.81	0.81
query15	0.79	0.76	0.76
query16	0.38	0.36	0.37
query17	1.04	1.03	1.01
query18	0.23	0.25	0.24
query19	1.97	1.82	1.83
query20	0.02	0.01	0.01
query21	15.51	0.54	0.54
query22	2.27	2.63	1.53
query23	17.16	0.84	0.87
query24	6.10	0.83	1.70
query25	0.39	0.12	0.04
query26	0.74	0.16	0.15
query27	0.04	0.04	0.04
query28	6.46	0.72	0.73
query29	12.61	2.37	2.32
query30	0.58	0.52	0.53
query31	2.79	0.38	0.39
query32	3.37	0.50	0.48
query33	3.09	3.11	3.03
query34	15.63	4.81	4.76
query35	4.84	4.85	4.85
query36	1.08	1.00	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.17	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.21 s
Total hot run time: 30.35 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 218339d097d18d369bd0bc97bcf4e2fc6c9919c0 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.6 seconds inserted 10000000 Rows, about 462K ops/s

@xiaokang xiaokang merged commit 08c13b0 into apache:branch-2.0 Jun 23, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants