Skip to content

Conversation

@yujun777
Copy link
Contributor

@yujun777 yujun777 commented Jul 9, 2024

cherry-pick: #37488, #33574

deardeng and others added 3 commits July 9, 2024 10:23
…pache#37488)

Fix: for creating table, if create dynamic partition throw exception (pr
apache#35778), then create table will fail, then drop table, but it forget to
write an editlog.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777
Copy link
Contributor Author

yujun777 commented Jul 9, 2024

run buildall

@yujun777 yujun777 changed the title [fix](create table) fix create table fail cause meta leaky #37488, #33574 [fix](create table) fix create table fail cause meta leaky #37488, #33574 Jul 9, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 49895 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 64290ffbf1b359ca77a868496a7c61e4cebd267a, data reload: false

------ Round 1 ----------------------------------
q1	17847	4449	4340	4340
q2	2092	157	148	148
q3	10435	1947	1964	1947
q4	10200	1251	1340	1251
q5	8527	3866	3938	3866
q6	235	124	146	124
q7	2054	1640	1611	1611
q8	9281	2742	2720	2720
q9	10695	10462	10245	10245
q10	8665	3540	3512	3512
q11	428	245	236	236
q12	462	306	310	306
q13	18348	3981	3999	3981
q14	361	333	340	333
q15	510	463	459	459
q16	667	573	575	573
q17	1144	936	982	936
q18	7210	6876	7007	6876
q19	1787	1617	1589	1589
q20	551	299	307	299
q21	4534	4147	4108	4108
q22	522	435	447	435
Total cold run time: 116555 ms
Total hot run time: 49895 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4310	4277	4305	4277
q2	328	227	220	220
q3	4213	4149	4142	4142
q4	2781	2756	2759	2756
q5	7148	7160	7127	7127
q6	239	117	116	116
q7	3256	2863	2844	2844
q8	4365	4465	4532	4465
q9	16835	16706	16818	16706
q10	4243	4311	4256	4256
q11	761	684	696	684
q12	1031	887	880	880
q13	7141	3716	3706	3706
q14	472	429	428	428
q15	504	456	455	455
q16	743	684	677	677
q17	3826	3910	3799	3799
q18	8862	8824	8788	8788
q19	1740	1700	1662	1662
q20	2375	2125	2142	2125
q21	8588	8665	8640	8640
q22	995	954	965	954
Total cold run time: 84756 ms
Total hot run time: 79707 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203429 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 64290ffbf1b359ca77a868496a7c61e4cebd267a, data reload: false

query1	928	418	375	375
query2	6528	2877	2678	2678
query3	6923	210	204	204
query4	20742	18124	18103	18103
query5	19726	6534	6452	6452
query6	284	219	228	219
query7	4159	297	317	297
query8	421	423	422	422
query9	3040	2627	2569	2569
query10	415	307	299	299
query11	11504	10662	10745	10662
query12	127	85	76	76
query13	5599	696	691	691
query14	17889	13642	13382	13382
query15	364	253	239	239
query16	6473	298	264	264
query17	1702	1447	891	891
query18	2308	417	407	407
query19	218	147	153	147
query20	80	80	80	80
query21	206	100	101	100
query22	5350	5235	5180	5180
query23	32389	32114	31872	31872
query24	7074	6499	6520	6499
query25	538	420	429	420
query26	541	162	162	162
query27	1909	291	295	291
query28	6103	2336	2311	2311
query29	3004	2678	2578	2578
query30	249	167	168	167
query31	906	734	745	734
query32	65	69	64	64
query33	402	269	250	250
query34	848	467	488	467
query35	1119	929	945	929
query36	1313	1362	1102	1102
query37	90	63	63	63
query38	3111	2903	2940	2903
query39	1389	1324	1307	1307
query40	198	89	94	89
query41	47	44	45	44
query42	87	87	83	83
query43	748	736	683	683
query44	1147	726	730	726
query45	247	235	232	232
query46	1226	956	982	956
query47	1878	1692	1694	1692
query48	1014	707	697	697
query49	623	383	365	365
query50	860	614	590	590
query51	4759	4647	4608	4608
query52	97	77	87	77
query53	448	311	324	311
query54	2647	2460	2461	2460
query55	97	81	92	81
query56	261	204	210	204
query57	1156	1132	1096	1096
query58	203	220	208	208
query59	4304	3989	4316	3989
query60	222	209	220	209
query61	100	95	98	95
query62	808	464	561	464
query63	487	351	347	347
query64	2509	1538	1405	1405
query65	3619	3572	3560	3560
query66	781	384	368	368
query67	16072	15830	15229	15229
query68	9284	635	674	635
query69	573	346	350	346
query70	1597	1395	1378	1378
query71	412	323	329	323
query72	6533	3495	3513	3495
query73	732	327	325	325
query74	6430	5890	5870	5870
query75	5361	3638	3718	3638
query76	5938	1131	1212	1131
query77	1010	263	266	263
query78	12545	11881	11417	11417
query79	9698	642	649	642
query80	1515	404	397	397
query81	504	234	238	234
query82	1705	102	96	96
query83	177	138	137	137
query84	257	72	76	72
query85	886	336	340	336
query86	330	308	310	308
query87	3277	3056	3032	3032
query88	5046	2334	2357	2334
query89	486	284	283	283
query90	1989	218	211	211
query91	178	147	139	139
query92	57	60	52	52
query93	6908	584	581	581
query94	706	215	209	209
query95	1105	1067	1059	1059
query96	653	331	341	331
query97	6467	6438	6469	6438
query98	200	181	167	167
query99	3073	883	826	826
Total cold run time: 318313 ms
Total hot run time: 203429 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 64290ffbf1b359ca77a868496a7c61e4cebd267a, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.06
query4	1.77	0.10	0.10
query5	0.53	0.52	0.53
query6	1.23	0.62	0.64
query7	0.01	0.02	0.02
query8	0.03	0.03	0.02
query9	0.51	0.48	0.47
query10	0.54	0.55	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.62
query14	0.78	0.78	0.77
query15	0.79	0.76	0.76
query16	0.35	0.34	0.38
query17	1.01	1.02	1.01
query18	0.21	0.27	0.22
query19	1.92	1.84	1.77
query20	0.01	0.02	0.02
query21	15.48	0.54	0.55
query22	2.13	2.23	1.99
query23	16.89	0.89	1.05
query24	7.99	0.69	0.93
query25	0.39	0.10	0.05
query26	0.79	0.16	0.15
query27	0.04	0.04	0.04
query28	5.58	0.75	0.72
query29	12.75	2.35	2.32
query30	0.57	0.53	0.52
query31	2.81	0.39	0.37
query32	3.40	0.50	0.48
query33	3.04	3.08	3.07
query34	15.25	4.81	4.78
query35	4.82	4.86	4.85
query36	1.06	1.01	1.01
query37	0.05	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.25 s
Total hot run time: 30.76 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 64290ffbf1b359ca77a868496a7c61e4cebd267a with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@dataroaring dataroaring merged commit 6789a56 into apache:branch-2.0 Jul 9, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants