Skip to content

[fix](coldhot) fix cannot cancel storage policy of partition #35885

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
lide-reed merged 1 commit into from
Jun 5, 2024
Merged

[fix](coldhot) fix cannot cancel storage policy of partition #35885

lide-reed merged 1 commit into from
Jun 5, 2024

Conversation

@Yulei-Yang
Copy link
Contributor

@Yulei-Yang Yulei-Yang commented Jun 5, 2024

bp: #35874

@doris-robot
Copy link

doris-robot commented Jun 5, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yulei-Yang
Copy link
Contributor Author

Yulei-Yang commented Jun 5, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 5, 2024

TPC-H: Total hot run time: 49624 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fe9100c6d2a0f277f2eac0996153b74514651bee, data reload: false

------ Round 1 ----------------------------------
q1	17860	4401	4356	4356
q2	2037	151	145	145
q3	10350	1921	1957	1921
q4	10318	1249	1317	1249
q5	8431	3867	3879	3867
q6	240	124	124	124
q7	2062	1592	1601	1592
q8	9518	2714	2685	2685
q9	13919	10314	10248	10248
q10	8671	3508	3497	3497
q11	415	238	255	238
q12	478	301	301	301
q13	18363	3965	4062	3965
q14	356	325	332	325
q15	509	457	462	457
q16	665	577	562	562
q17	1122	921	903	903
q18	7271	6834	7005	6834
q19	1699	1596	1579	1579
q20	551	319	288	288
q21	4377	4184	4095	4095
q22	501	393	394	393
Total cold run time: 119713 ms
Total hot run time: 49624 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4311	4277	4315	4277
q2	328	230	223	223
q3	4185	4126	4145	4126
q4	2780	2748	2746	2746
q5	7089	7048	7127	7048
q6	240	121	122	121
q7	3230	2823	2771	2771
q8	4328	4444	4483	4444
q9	16923	16741	16649	16649
q10	4200	4264	4271	4264
q11	781	676	653	653
q12	1046	834	869	834
q13	6711	3756	3712	3712
q14	472	431	429	429
q15	496	458	456	456
q16	720	679	670	670
q17	3831	3848	3812	3812
q18	8747	8739	8791	8739
q19	1694	1662	1625	1625
q20	2396	2135	2105	2105
q21	8485	8420	8449	8420
q22	1031	928	949	928
Total cold run time: 84024 ms
Total hot run time: 79052 ms

@doris-robot
Copy link

doris-robot commented Jun 5, 2024

TPC-DS: Total hot run time: 203408 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fe9100c6d2a0f277f2eac0996153b74514651bee, data reload: false

query1	948	436	378	378
query2	6530	2912	2484	2484
query3	6920	216	209	209
query4	20660	17940	18011	17940
query5	19720	6530	6445	6445
query6	285	212	236	212
query7	4163	306	316	306
query8	256	234	252	234
query9	3176	2730	2681	2681
query10	415	296	291	291
query11	11390	10708	10727	10708
query12	127	84	77	77
query13	5599	665	690	665
query14	17708	13702	13646	13646
query15	371	235	230	230
query16	6488	289	267	267
query17	1706	1442	912	912
query18	2297	415	409	409
query19	211	155	150	150
query20	81	77	83	77
query21	194	95	95	95
query22	5389	5021	5021	5021
query23	32788	32120	31911	31911
query24	7040	6526	6456	6456
query25	518	438	431	431
query26	647	170	167	167
query27	2030	305	304	304
query28	6134	2335	2299	2299
query29	2871	2789	2733	2733
query30	245	172	173	172
query31	916	740	750	740
query32	73	66	59	59
query33	390	266	262	262
query34	867	482	488	482
query35	1120	965	940	940
query36	1515	1141	1079	1079
query37	90	59	59	59
query38	3091	2907	2937	2907
query39	1385	1345	1327	1327
query40	302	95	94	94
query41	41	38	37	37
query42	86	89	86	86
query43	792	745	613	613
query44	1129	710	717	710
query45	243	227	230	227
query46	1226	975	989	975
query47	1983	1731	1777	1731
query48	1007	741	716	716
query49	631	386	376	376
query50	857	654	611	611
query51	4839	4660	4652	4652
query52	99	84	83	83
query53	456	327	322	322
query54	2661	2493	2484	2484
query55	99	85	84	84
query56	250	224	216	216
query57	1156	1125	1103	1103
query58	226	206	213	206
query59	4010	3878	3984	3878
query60	229	207	228	207
query61	101	96	102	96
query62	863	426	447	426
query63	501	346	370	346
query64	2648	1544	1456	1456
query65	3615	3577	3606	3577
query66	843	383	379	379
query67	16259	15157	15750	15157
query68	10981	679	658	658
query69	580	339	363	339
query70	2114	1483	1360	1360
query71	422	297	318	297
query72	6578	3479	3524	3479
query73	750	333	319	319
query74	6348	5862	5864	5862
query75	5557	3763	3621	3621
query76	6948	1146	1207	1146
query77	1183	259	260	259
query78	12546	11875	11688	11688
query79	7961	634	652	634
query80	1099	394	396	394
query81	461	232	248	232
query82	778	99	98	98
query83	171	143	134	134
query84	259	75	70	70
query85	873	319	315	315
query86	346	294	292	292
query87	3243	3045	3001	3001
query88	4188	2345	2349	2345
query89	471	279	294	279
query90	1929	213	225	213
query91	178	140	141	140
query92	59	53	51	51
query93	7054	578	576	576
query94	700	210	225	210
query95	1117	1073	1043	1043
query96	654	326	329	326
query97	6518	6330	6316	6316
query98	187	177	173	173
query99	3117	939	925	925
Total cold run time: 318876 ms
Total hot run time: 203408 ms

@doris-robot
Copy link

doris-robot commented Jun 5, 2024

ClickBench: Total hot run time: 30.69 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fe9100c6d2a0f277f2eac0996153b74514651bee, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.24	0.04	0.04
query4	1.83	0.08	0.06
query5	0.54	0.53	0.51
query6	1.24	0.60	0.62
query7	0.01	0.00	0.01
query8	0.03	0.02	0.03
query9	0.54	0.47	0.47
query10	0.55	0.52	0.53
query11	0.13	0.08	0.09
query12	0.12	0.09	0.09
query13	0.63	0.62	0.61
query14	0.77	0.79	0.80
query15	0.78	0.76	0.77
query16	0.36	0.39	0.35
query17	1.03	0.98	1.04
query18	0.22	0.25	0.24
query19	1.90	1.84	1.91
query20	0.01	0.01	0.01
query21	15.44	0.57	0.54
query22	2.15	2.03	1.59
query23	16.48	1.01	1.16
query24	4.90	3.67	1.01
query25	0.38	0.13	0.05
query26	0.77	0.15	0.16
query27	0.04	0.04	0.03
query28	5.61	0.78	0.71
query29	12.67	2.22	2.26
query30	0.53	0.52	0.52
query31	2.81	0.38	0.38
query32	3.40	0.50	0.49
query33	3.12	3.07	3.08
query34	15.25	4.81	4.81
query35	4.88	4.84	4.85
query36	1.04	1.02	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.17	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 100.89 s
Total hot run time: 30.69 s

@doris-robot
Copy link

doris-robot commented Jun 5, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit fe9100c6d2a0f277f2eac0996153b74514651bee with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

lide-reed
lide-reed approved these changes Jun 5, 2024
View reviewed changes
Copy link
Contributor

@lide-reed lide-reed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 5, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2024

PR approved by anyone and no changes requested.

@lide-reed lide-reed merged commit 18fd1cf into apache:branch-2.0 Jun 5, 2024
@Yulei-Yang Yulei-Yang deleted the fix_cancel_partition_storage_policy_for_v2 branch June 5, 2024 07:53
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@Yulei-Yang
[fix](coldhot) fix cannot cancel storage policy of partition (apache#…
63dbdc8 
…35885)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lide-reed lide-reed lide-reed approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. kind/test reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Yulei-Yang @doris-robot @lide-reed