Skip to content

[fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange (#38688) #38881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
w41ter merged 1 commit into from
Aug 5, 2024
Merged

[fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange (#38688) #38881

w41ter merged 1 commit into from
Aug 5, 2024

Conversation

@w41ter
Copy link
Contributor

@w41ter w41ter commented Aug 5, 2024

No description provided.

@w41ter
[fix](fe) Fix the default value of ReplacePartitionClause.isStrictRan…
16d2c8c 
…ge (apache#38688)

During ReplacePartitionClause.analyze(), the default value of
isStrictRange is true. However, ReplacePartitionClause instances
constructed by internal code do not call analyze(), so their
isStrictRange value is incorrect (e.g., INSERT INTO ... OVERWRITE).

This PR handle the relevant properties when constructing
ReplacePartitionClause.
@doris-robot
Copy link

doris-robot commented Aug 5, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Aug 5, 2024

run buildall

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Aug 5, 2024
@doris-robot
Copy link

doris-robot commented Aug 5, 2024

TPC-H: Total hot run time: 49804 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16d2c8cdb3944e58299f405a23db7e1b7069ede7, data reload: false

------ Round 1 ----------------------------------
q1	18079	4421	4392	4392
q2	2066	157	159	157
q3	10260	1919	1934	1919
q4	10111	1282	1287	1282
q5	8540	3898	3924	3898
q6	236	127	149	127
q7	2046	1637	1630	1630
q8	9563	2728	2711	2711
q9	13634	10225	10214	10214
q10	8697	3537	3527	3527
q11	419	251	247	247
q12	472	305	306	305
q13	18366	3949	4032	3949
q14	363	328	320	320
q15	511	465	456	456
q16	664	571	588	571
q17	1134	904	867	867
q18	7350	6873	6806	6806
q19	1768	1692	1663	1663
q20	550	309	306	306
q21	4448	4096	4011	4011
q22	542	446	452	446
Total cold run time: 119819 ms
Total hot run time: 49804 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4305	4298	4261	4261
q2	325	231	223	223
q3	4163	4145	4117	4117
q4	2756	2742	2722	2722
q5	7217	7107	7126	7107
q6	236	120	121	120
q7	3267	2804	2836	2804
q8	4407	4476	4482	4476
q9	16825	16753	16724	16724
q10	4234	4294	4259	4259
q11	775	707	697	697
q12	1032	867	846	846
q13	5570	3738	3717	3717
q14	448	428	431	428
q15	503	453	450	450
q16	738	687	687	687
q17	3829	3865	3850	3850
q18	8768	8743	8787	8743
q19	1730	1679	1678	1678
q20	2358	2156	2157	2156
q21	8544	8591	8555	8555
q22	1074	1001	1003	1001
Total cold run time: 83104 ms
Total hot run time: 79621 ms

@doris-robot
Copy link

doris-robot commented Aug 5, 2024

TPC-DS: Total hot run time: 204441 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16d2c8cdb3944e58299f405a23db7e1b7069ede7, data reload: false

query1	935	386	412	386
query2	6577	2706	2590	2590
query3	6925	208	204	204
query4	20918	18126	17930	17930
query5	19735	6542	6573	6542
query6	287	224	237	224
query7	4167	297	314	297
query8	445	447	409	409
query9	3075	2629	2590	2590
query10	423	292	312	292
query11	11265	10755	10704	10704
query12	130	74	74	74
query13	5617	697	698	697
query14	17813	13762	13421	13421
query15	362	236	254	236
query16	6457	290	258	258
query17	1710	1447	884	884
query18	2301	423	424	423
query19	212	152	152	152
query20	74	79	78	78
query21	190	100	93	93
query22	5381	5198	5082	5082
query23	32568	31962	31927	31927
query24	6903	6540	6540	6540
query25	518	453	439	439
query26	528	163	165	163
query27	1893	299	303	299
query28	6164	2343	2303	2303
query29	2779	2747	2735	2735
query30	244	169	168	168
query31	887	730	781	730
query32	71	60	59	59
query33	402	258	268	258
query34	855	481	489	481
query35	1125	941	908	908
query36	1358	1394	1123	1123
query37	90	61	62	61
query38	3093	2931	2929	2929
query39	1358	1337	1309	1309
query40	214	92	99	92
query41	46	43	46	43
query42	82	91	89	89
query43	785	705	799	705
query44	1154	721	716	716
query45	250	234	236	234
query46	1228	946	974	946
query47	1720	1654	1621	1621
query48	994	723	722	722
query49	622	364	375	364
query50	854	648	622	622
query51	4741	4796	4666	4666
query52	89	87	79	79
query53	453	330	321	321
query54	2667	2479	2446	2446
query55	98	75	89	75
query56	249	216	217	216
query57	1253	1094	1172	1094
query58	216	201	212	201
query59	4269	4078	3617	3617
query60	214	202	222	202
query61	97	92	96	92
query62	783	511	481	481
query63	495	349	353	349
query64	2385	1538	1503	1503
query65	3635	3571	3558	3558
query66	784	376	397	376
query67	15302	16753	16496	16496
query68	8647	632	657	632
query69	589	342	355	342
query70	1821	1574	1371	1371
query71	408	301	316	301
query72	6564	3499	3460	3460
query73	729	324	322	322
query74	6264	5906	5800	5800
query75	5350	3691	3685	3685
query76	5215	1165	1175	1165
query77	906	266	250	250
query78	12599	11546	11457	11457
query79	8795	621	646	621
query80	1304	404	401	401
query81	495	236	234	234
query82	1738	96	98	96
query83	171	130	131	130
query84	261	71	70	70
query85	892	332	333	332
query86	337	291	293	291
query87	3193	2972	3077	2972
query88	5121	2302	2311	2302
query89	387	285	305	285
query90	1912	209	215	209
query91	179	143	151	143
query92	68	52	52	52
query93	5667	581	597	581
query94	726	210	207	207
query95	1101	1058	1045	1045
query96	644	317	328	317
query97	6400	6429	6414	6414
query98	192	176	177	176
query99	2941	854	897	854
Total cold run time: 313130 ms
Total hot run time: 204441 ms

@doris-robot
Copy link

doris-robot commented Aug 5, 2024

ClickBench: Total hot run time: 31.18 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16d2c8cdb3944e58299f405a23db7e1b7069ede7, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.78	0.07	0.07
query5	0.54	0.53	0.51
query6	1.23	0.62	0.61
query7	0.02	0.01	0.00
query8	0.04	0.03	0.02
query9	0.52	0.50	0.48
query10	0.54	0.54	0.53
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.61	0.61	0.61
query14	0.80	0.80	0.77
query15	0.78	0.76	0.76
query16	0.38	0.37	0.37
query17	1.01	0.98	1.01
query18	0.24	0.25	0.25
query19	1.94	1.85	1.86
query20	0.02	0.01	0.01
query21	15.45	0.56	0.54
query22	1.98	2.18	1.61
query23	17.09	1.00	0.97
query24	4.88	1.96	1.43
query25	0.36	0.12	0.04
query26	0.69	0.16	0.15
query27	0.04	0.05	0.04
query28	6.86	0.72	0.75
query29	12.84	2.32	2.38
query30	0.56	0.54	0.47
query31	2.80	0.38	0.38
query32	3.38	0.50	0.50
query33	3.09	3.05	3.03
query34	15.28	4.79	4.77
query35	4.83	4.87	4.86
query36	1.07	1.03	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 102.61 s
Total hot run time: 31.18 s

@doris-robot
Copy link

doris-robot commented Aug 5, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 16d2c8cdb3944e58299f405a23db7e1b7069ede7 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@w41ter w41ter merged commit 3b1bb2e into apache:branch-2.0 Aug 5, 2024
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@w41ter
[fix](fe) Fix the default value of ReplacePartitionClause.isStrictRan…
75d4d4d 
…ge (apache#38688) (apache#38881)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/planner Issues or PRs related to the query planner

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@w41ter @doris-robot