Skip to content

Conversation

@mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Jul 10, 2024

Proposed changes

Fix some problems found by https://github.com/apache/doris/pull/26856/files

  1. Some insert should not run into group commit mode, including insert overwrite and mvmt
  2. If there is no partition for the data, should throw an exception

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jul 10, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39889 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ffbfb12cb7cd4ba8da37076e9804e5d1791136b0, data reload: false

------ Round 1 ----------------------------------
q1	17593	4556	4254	4254
q2	2026	193	188	188
q3	10436	1230	1176	1176
q4	10184	857	746	746
q5	7612	2704	2664	2664
q6	227	141	146	141
q7	982	616	631	616
q8	9285	2084	2071	2071
q9	8975	6533	6493	6493
q10	8897	3786	3690	3690
q11	459	237	251	237
q12	412	233	240	233
q13	17770	3026	2992	2992
q14	280	231	229	229
q15	530	493	500	493
q16	512	384	388	384
q17	960	644	743	644
q18	8241	7433	7505	7433
q19	1661	1538	1373	1373
q20	686	327	338	327
q21	5037	3169	3353	3169
q22	387	340	336	336
Total cold run time: 113152 ms
Total hot run time: 39889 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4397	4242	4425	4242
q2	369	267	265	265
q3	2974	2787	2736	2736
q4	1899	1639	1626	1626
q5	5327	5298	5310	5298
q6	227	133	133	133
q7	2115	1791	1803	1791
q8	3271	3344	3340	3340
q9	8433	8397	8333	8333
q10	3895	3690	3649	3649
q11	581	495	476	476
q12	803	597	615	597
q13	17660	3029	2982	2982
q14	299	269	276	269
q15	514	476	470	470
q16	479	401	424	401
q17	1757	1500	1454	1454
q18	7668	7571	7379	7379
q19	8928	1737	1575	1575
q20	2040	1753	1782	1753
q21	4927	4806	4750	4750
q22	610	539	551	539
Total cold run time: 79173 ms
Total hot run time: 54058 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174192 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ffbfb12cb7cd4ba8da37076e9804e5d1791136b0, data reload: false

query1	930	370	365	365
query2	6447	2480	2286	2286
query3	6657	221	214	214
query4	28499	17323	17434	17323
query5	4209	479	476	476
query6	291	172	195	172
query7	4586	304	292	292
query8	330	283	295	283
query9	8561	2391	2369	2369
query10	447	288	276	276
query11	10720	10067	10347	10067
query12	137	88	84	84
query13	1650	391	380	380
query14	9578	8455	7817	7817
query15	271	182	203	182
query16	7370	328	315	315
query17	1784	573	570	570
query18	1921	299	286	286
query19	201	158	161	158
query20	89	86	82	82
query21	203	131	134	131
query22	4451	4085	4093	4085
query23	33807	33178	33273	33178
query24	11904	2894	2812	2812
query25	659	391	396	391
query26	1625	159	164	159
query27	2684	273	280	273
query28	7433	2080	2068	2068
query29	950	655	637	637
query30	280	150	151	150
query31	958	725	767	725
query32	95	59	58	58
query33	792	314	310	310
query34	893	489	504	489
query35	687	568	586	568
query36	1085	953	935	935
query37	159	83	84	83
query38	2875	2736	2782	2736
query39	902	816	816	816
query40	284	128	132	128
query41	55	55	60	55
query42	119	103	105	103
query43	613	566	544	544
query44	1243	742	744	742
query45	191	164	164	164
query46	1071	729	717	717
query47	1853	1773	1764	1764
query48	383	311	312	311
query49	1187	436	429	429
query50	776	400	408	400
query51	6972	6727	6791	6727
query52	105	102	96	96
query53	364	297	294	294
query54	1012	470	465	465
query55	77	79	78	78
query56	325	293	306	293
query57	1168	1083	1060	1060
query58	273	351	248	248
query59	3734	3445	3191	3191
query60	305	277	281	277
query61	95	95	95	95
query62	829	647	670	647
query63	322	305	291	291
query64	10628	2181	1631	1631
query65	3185	3128	3108	3108
query66	1382	338	340	338
query67	15738	15031	14920	14920
query68	8442	566	567	566
query69	735	475	360	360
query70	1245	1156	1105	1105
query71	540	289	294	289
query72	9095	5217	5528	5217
query73	826	334	326	326
query74	5994	5538	5441	5441
query75	5191	2699	2682	2682
query76	5121	933	926	926
query77	778	316	313	313
query78	10532	8888	8937	8888
query79	9391	523	529	523
query80	1267	495	498	495
query81	586	222	217	217
query82	443	136	138	136
query83	337	171	173	171
query84	280	90	88	88
query85	1086	327	350	327
query86	354	311	312	311
query87	3310	3160	3121	3121
query88	4434	2449	2477	2449
query89	497	395	384	384
query90	2130	190	200	190
query91	138	105	105	105
query92	64	50	52	50
query93	6713	525	524	524
query94	1440	217	211	211
query95	404	327	324	324
query96	633	273	268	268
query97	3168	3041	3001	3001
query98	222	207	193	193
query99	1555	1281	1274	1274
Total cold run time: 308360 ms
Total hot run time: 174192 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ffbfb12cb7cd4ba8da37076e9804e5d1791136b0, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.07
query5	0.50	0.48	0.48
query6	1.15	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.56	0.51	0.50
query10	0.55	0.56	0.56
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.59
query14	0.77	0.78	0.78
query15	0.86	0.81	0.82
query16	0.37	0.36	0.36
query17	1.01	1.04	1.06
query18	0.24	0.22	0.22
query19	1.86	1.86	1.68
query20	0.02	0.01	0.01
query21	15.40	0.77	0.66
query22	4.43	8.20	1.51
query23	18.27	1.36	1.36
query24	2.07	0.22	0.22
query25	0.16	0.09	0.09
query26	0.29	0.22	0.21
query27	0.45	0.22	0.22
query28	13.36	1.01	1.02
query29	12.64	3.36	3.33
query30	0.25	0.06	0.05
query31	2.90	0.38	0.38
query32	3.24	0.48	0.46
query33	2.88	2.93	2.91
query34	17.38	4.40	4.35
query35	4.44	4.44	4.41
query36	0.63	0.48	0.47
query37	0.19	0.15	0.16
query38	0.16	0.14	0.15
query39	0.04	0.03	0.03
query40	0.16	0.12	0.12
query41	0.10	0.06	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.42 s
Total hot run time: 30.43 s

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Jul 10, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39899 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 31ada78775263e8699df7d0af861506cb50c5fc8, data reload: false

------ Round 1 ----------------------------------
q1	17617	4342	4247	4247
q2	2016	200	187	187
q3	10500	1135	1004	1004
q4	10203	831	842	831
q5	7483	2671	2637	2637
q6	217	144	140	140
q7	952	604	592	592
q8	9304	2073	2068	2068
q9	8892	6542	6485	6485
q10	9034	3730	3699	3699
q11	455	245	248	245
q12	450	234	234	234
q13	17773	2976	3007	2976
q14	270	226	232	226
q15	535	491	497	491
q16	502	385	380	380
q17	954	633	691	633
q18	8014	7452	7561	7452
q19	6822	1477	1502	1477
q20	658	326	333	326
q21	4953	3223	3999	3223
q22	389	346	351	346
Total cold run time: 117993 ms
Total hot run time: 39899 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4258	4275	4258
q2	374	269	269	269
q3	2948	2940	2969	2940
q4	1969	1736	1784	1736
q5	5549	5501	5478	5478
q6	232	136	141	136
q7	2272	1905	1926	1905
q8	3229	3455	3421	3421
q9	8685	8878	8799	8799
q10	4122	3771	3852	3771
q11	591	483	497	483
q12	824	639	634	634
q13	16267	3149	3164	3149
q14	297	287	272	272
q15	545	488	482	482
q16	491	432	445	432
q17	1817	1560	1504	1504
q18	8128	7948	7942	7942
q19	1792	1628	1720	1628
q20	2969	1869	1872	1869
q21	9831	4779	4906	4779
q22	690	555	545	545
Total cold run time: 78017 ms
Total hot run time: 56432 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174547 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 31ada78775263e8699df7d0af861506cb50c5fc8, data reload: false

query1	910	376	380	376
query2	6363	2485	2440	2440
query3	6672	202	212	202
query4	25775	17402	17383	17383
query5	3639	485	469	469
query6	269	178	180	178
query7	4583	288	288	288
query8	308	287	294	287
query9	8590	2372	2356	2356
query10	428	291	273	273
query11	12597	9994	10029	9994
query12	123	83	86	83
query13	1657	397	395	395
query14	10075	7674	7925	7674
query15	246	201	196	196
query16	7784	321	321	321
query17	1578	561	534	534
query18	1958	291	284	284
query19	211	157	161	157
query20	95	85	86	85
query21	199	130	130	130
query22	4211	4084	4180	4084
query23	33954	33822	33540	33540
query24	11329	2868	2907	2868
query25	646	470	389	389
query26	953	153	156	153
query27	2375	270	274	270
query28	6479	2168	2129	2129
query29	896	643	685	643
query30	246	151	155	151
query31	972	758	746	746
query32	93	58	54	54
query33	760	295	295	295
query34	1005	487	490	487
query35	684	577	601	577
query36	1136	1023	959	959
query37	150	88	83	83
query38	2985	2860	2820	2820
query39	893	856	818	818
query40	204	119	118	118
query41	46	48	53	48
query42	113	100	100	100
query43	606	529	567	529
query44	1227	735	734	734
query45	196	160	162	160
query46	1072	726	725	725
query47	1837	1792	1765	1765
query48	365	303	300	300
query49	842	415	427	415
query50	792	393	396	393
query51	7012	6844	6770	6770
query52	109	89	94	89
query53	359	289	296	289
query54	908	451	450	450
query55	74	74	74	74
query56	280	280	278	278
query57	1131	1082	1031	1031
query58	259	243	261	243
query59	3330	3315	3266	3266
query60	312	289	285	285
query61	99	97	98	97
query62	816	654	651	651
query63	325	296	297	296
query64	9438	2192	1625	1625
query65	3136	3096	3121	3096
query66	752	325	331	325
query67	15591	15185	14990	14990
query68	4565	541	541	541
query69	612	425	350	350
query70	1210	1097	1098	1097
query71	444	280	279	279
query72	8188	5176	5613	5176
query73	741	327	323	323
query74	5836	5508	5543	5508
query75	4140	2681	2629	2629
query76	3209	901	904	901
query77	690	306	304	304
query78	9512	8933	8899	8899
query79	3036	527	520	520
query80	1622	468	475	468
query81	589	222	218	218
query82	888	135	135	135
query83	337	180	174	174
query84	274	88	94	88
query85	1991	313	304	304
query86	486	304	302	302
query87	3300	3080	3187	3080
query88	4163	2445	2422	2422
query89	489	386	390	386
query90	1877	188	192	188
query91	137	105	110	105
query92	68	50	51	50
query93	5090	524	515	515
query94	1185	215	212	212
query95	405	317	320	317
query96	612	276	275	275
query97	3248	3021	3035	3021
query98	217	208	192	192
query99	1507	1292	1255	1255
Total cold run time: 285726 ms
Total hot run time: 174547 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 31ada78775263e8699df7d0af861506cb50c5fc8, data reload: false

query1	0.04	0.04	0.02
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.69	0.10	0.09
query5	0.49	0.49	0.48
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.48	0.48
query10	0.55	0.54	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.58	0.58
query14	0.76	0.78	0.77
query15	0.86	0.82	0.81
query16	0.36	0.35	0.37
query17	1.01	0.97	1.00
query18	0.23	0.23	0.22
query19	1.79	1.74	1.71
query20	0.01	0.00	0.01
query21	15.39	0.76	0.66
query22	4.22	7.30	2.12
query23	18.28	1.44	1.25
query24	2.16	0.23	0.22
query25	0.15	0.09	0.08
query26	0.30	0.21	0.21
query27	0.46	0.23	0.24
query28	13.22	1.02	1.00
query29	12.59	3.29	3.33
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.28	0.48	0.45
query33	2.90	2.92	2.86
query34	16.81	4.31	4.44
query35	4.48	4.45	4.44
query36	0.65	0.48	0.48
query37	0.19	0.15	0.16
query38	0.15	0.14	0.15
query39	0.05	0.03	0.04
query40	0.16	0.12	0.13
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.49 s
Total hot run time: 30.73 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 11, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 24969d5 into apache:master Jul 11, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
## Proposed changes

Fix some problems found by
https://github.com/apache/doris/pull/26856/files

1. Some insert should not run into group commit mode, including insert
overwrite and mvmt
2. If there is no partition for the data, should throw an exception
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
## Proposed changes

Fix some problems found by
https://github.com/apache/doris/pull/26856/files

1. Some insert should not run into group commit mode, including insert
overwrite and mvmt
2. If there is no partition for the data, should throw an exception
dataroaring pushed a commit that referenced this pull request Jul 25, 2024
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants