Skip to content

[enhance](mtmv)show create materialized view #36188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 28 commits into from
Jun 21, 2024
Merged

[enhance](mtmv)show create materialized view #36188

morrySnow merged 28 commits into from
Jun 21, 2024

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented Jun 12, 2024 •
edited
Loading

  • support show create materialized view
  • Prohibit show create table from specifying MTMV
 mysql> show create MATERIALIZED VIEW mv1\G
*************************** 1. row ***************************
       Materialized View: mv1
Create Materialized View: CREATE MATERIALIZED VIEW mv1
(k1,k2)
BUILD DEFERRED REFRESH AUTO ON MANUAL
DUPLICATE KEY(`k1`, `k2`)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
)
AS SELECT * from user1
1 row in set (0.01 sec)

mysql> show create table rollup_mv;
ERROR 1105 (HY000): errCode = 2, detailMessage = not support async materialized view, please use `show create materialized view`

doc: apache/doris-website#870

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr
Copy link
Contributor Author

zddr commented Jun 12, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-H: Total hot run time: 40203 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 19cd0df0255afd772a3a10ebb55ea6f01530560e, data reload: false

------ Round 1 ----------------------------------
q1	17614	4311	4251	4251
q2	2014	186	192	186
q3	10458	1123	1176	1123
q4	10189	792	753	753
q5	7436	2663	2590	2590
q6	219	136	135	135
q7	959	614	585	585
q8	9224	2052	2068	2052
q9	8905	6534	6486	6486
q10	9087	3717	3695	3695
q11	453	237	235	235
q12	497	227	222	222
q13	18999	3013	3015	3013
q14	259	214	229	214
q15	523	475	482	475
q16	527	402	390	390
q17	967	660	676	660
q18	8327	8034	7875	7875
q19	7981	1380	1541	1380
q20	696	333	326	326
q21	5324	4389	3207	3207
q22	423	365	350	350
Total cold run time: 121081 ms
Total hot run time: 40203 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4869	4596	4651	4596
q2	384	268	260	260
q3	3179	2993	3087	2993
q4	2023	1729	1739	1729
q5	5676	5586	5611	5586
q6	242	132	131	131
q7	2241	1846	1832	1832
q8	3367	3389	3445	3389
q9	8679	8650	8678	8650
q10	4092	3871	3884	3871
q11	597	514	491	491
q12	814	644	643	643
q13	17154	3010	3123	3010
q14	298	274	259	259
q15	523	498	491	491
q16	509	429	444	429
q17	1830	1555	1524	1524
q18	8046	7880	7840	7840
q19	1747	1545	1697	1545
q20	3351	1896	1847	1847
q21	10160	4642	4876	4642
q22	701	540	553	540
Total cold run time: 80482 ms
Total hot run time: 56298 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-DS: Total hot run time: 171651 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 19cd0df0255afd772a3a10ebb55ea6f01530560e, data reload: false

query1	806	385	381	381
query2	6389	2341	2391	2341
query3	6411	209	221	209
query4	18960	17488	17358	17358
query5	3722	464	470	464
query6	251	160	159	159
query7	4453	296	293	293
query8	300	291	297	291
query9	8284	2359	2340	2340
query10	552	289	270	270
query11	10549	9995	9920	9920
query12	134	87	85	85
query13	1570	364	373	364
query14	10012	6166	7113	6166
query15	228	194	186	186
query16	7272	266	272	266
query17	1389	575	556	556
query18	1920	268	272	268
query19	190	156	154	154
query20	90	79	78	78
query21	208	126	126	126
query22	4302	4102	3933	3933
query23	33445	33382	33580	33382
query24	8055	2806	2888	2806
query25	605	357	345	345
query26	705	147	146	146
query27	2222	316	323	316
query28	5670	2028	2031	2028
query29	874	640	609	609
query30	240	147	150	147
query31	955	738	724	724
query32	93	53	57	53
query33	659	276	268	268
query34	872	477	465	465
query35	704	622	601	601
query36	1084	934	929	929
query37	135	65	70	65
query38	2867	2679	2695	2679
query39	865	789	816	789
query40	201	124	128	124
query41	55	51	55	51
query42	119	99	94	94
query43	593	545	543	543
query44	1077	739	736	736
query45	191	164	166	164
query46	1071	714	699	699
query47	1836	1740	1780	1740
query48	365	300	291	291
query49	877	433	392	392
query50	761	382	374	374
query51	6642	6704	6609	6609
query52	99	91	91	91
query53	348	284	279	279
query54	724	477	447	447
query55	73	73	82	73
query56	276	263	259	259
query57	1118	1036	1016	1016
query58	254	233	255	233
query59	3449	3214	3269	3214
query60	294	266	268	266
query61	93	95	89	89
query62	604	452	448	448
query63	317	282	288	282
query64	8589	2245	1795	1795
query65	3158	3101	3108	3101
query66	904	322	335	322
query67	15213	14934	14894	14894
query68	4469	553	540	540
query69	455	288	290	288
query70	1158	1150	1123	1123
query71	372	275	272	272
query72	7121	5395	5284	5284
query73	742	329	324	324
query74	5923	5489	5493	5489
query75	3402	2700	2656	2656
query76	2252	966	918	918
query77	450	294	288	288
query78	10330	9917	9682	9682
query79	2671	502	497	497
query80	1233	476	448	448
query81	590	221	226	221
query82	787	100	102	100
query83	222	165	167	165
query84	240	85	83	83
query85	1644	279	271	271
query86	517	311	308	308
query87	3264	3079	3053	3053
query88	4302	2440	2464	2440
query89	463	389	373	373
query90	1909	188	186	186
query91	122	100	106	100
query92	61	54	46	46
query93	2125	502	503	502
query94	1259	182	257	182
query95	402	315	305	305
query96	599	275	274	274
query97	3163	3029	3052	3029
query98	214	208	196	196
query99	1172	836	822	822
Total cold run time: 260976 ms
Total hot run time: 171651 ms

morrySnow
morrySnow reviewed Jun 12, 2024
View reviewed changes
@doris-robot
Copy link

doris-robot commented Jun 12, 2024

ClickBench: Total hot run time: 30.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 19cd0df0255afd772a3a10ebb55ea6f01530560e, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.23	0.06	0.04
query4	1.66	0.06	0.06
query5	0.48	0.49	0.49
query6	1.12	0.71	0.72
query7	0.02	0.01	0.02
query8	0.06	0.04	0.03
query9	0.54	0.50	0.50
query10	0.55	0.55	0.54
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.59	0.59	0.62
query14	0.76	0.80	0.78
query15	0.83	0.81	0.81
query16	0.37	0.37	0.38
query17	1.05	1.02	0.97
query18	0.21	0.26	0.26
query19	1.79	1.79	1.72
query20	0.01	0.02	0.01
query21	15.41	0.66	0.65
query22	4.33	7.00	1.81
query23	18.28	1.39	1.30
query24	2.18	0.22	0.21
query25	0.14	0.09	0.09
query26	0.28	0.17	0.18
query27	0.08	0.08	0.07
query28	13.24	1.02	1.00
query29	12.64	3.24	3.21
query30	0.25	0.06	0.06
query31	2.84	0.39	0.38
query32	3.28	0.47	0.48
query33	2.85	2.87	2.89
query34	17.23	4.37	4.41
query35	4.50	4.52	4.53
query36	0.66	0.46	0.47
query37	0.18	0.16	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.04	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.63 s
Total hot run time: 30.41 s

@zddr
Copy link
Contributor Author

zddr commented Jun 12, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-H: Total hot run time: 39862 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51398770f37e6becf4cf3cfdb66ab128c3f47402, data reload: false

------ Round 1 ----------------------------------
q1	17624	4308	4281	4281
q2	2023	190	199	190
q3	10458	1113	1223	1113
q4	10183	858	774	774
q5	7461	2599	2605	2599
q6	223	139	137	137
q7	964	613	593	593
q8	9231	2052	2078	2052
q9	8893	6485	6500	6485
q10	8837	3754	3711	3711
q11	450	244	234	234
q12	413	245	219	219
q13	17762	2976	2973	2973
q14	274	225	229	225
q15	522	469	475	469
q16	516	389	375	375
q17	962	667	767	667
q18	8064	7672	7342	7342
q19	6368	1553	1564	1553
q20	651	320	325	320
q21	4933	3214	3220	3214
q22	416	337	336	336
Total cold run time: 117228 ms
Total hot run time: 39862 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4345	4218	4229	4218
q2	379	278	275	275
q3	3011	2800	2723	2723
q4	1882	1571	1608	1571
q5	5239	5260	5259	5259
q6	219	127	129	127
q7	2105	1761	1771	1761
q8	3183	3296	3315	3296
q9	8363	8319	8357	8319
q10	3905	3659	3679	3659
q11	614	496	486	486
q12	784	632	593	593
q13	16392	2997	3065	2997
q14	284	261	259	259
q15	525	480	474	474
q16	467	433	431	431
q17	1798	1512	1470	1470
q18	7721	7439	7322	7322
q19	1710	1648	1690	1648
q20	2009	1810	1783	1783
q21	4914	4827	4793	4793
q22	624	528	563	528
Total cold run time: 70473 ms
Total hot run time: 53992 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-DS: Total hot run time: 173591 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51398770f37e6becf4cf3cfdb66ab128c3f47402, data reload: false

query1	938	393	380	380
query2	6446	2517	2268	2268
query3	6653	211	214	211
query4	19215	17373	17428	17373
query5	4153	473	454	454
query6	247	167	161	161
query7	4597	311	298	298
query8	313	304	307	304
query9	8668	2373	2357	2357
query10	625	313	294	294
query11	10530	10114	10196	10114
query12	143	87	85	85
query13	1656	367	363	363
query14	9954	7463	7862	7463
query15	220	186	189	186
query16	7502	272	271	271
query17	1890	535	546	535
query18	1226	276	280	276
query19	197	157	153	153
query20	90	88	85	85
query21	216	135	122	122
query22	4296	4085	4024	4024
query23	33708	33169	33000	33000
query24	11997	2792	2796	2792
query25	674	351	363	351
query26	1635	158	151	151
query27	2687	330	321	321
query28	6705	2033	2027	2027
query29	1004	638	616	616
query30	288	151	159	151
query31	942	720	763	720
query32	90	56	57	56
query33	763	288	296	288
query34	934	473	469	469
query35	740	676	636	636
query36	1084	893	904	893
query37	187	71	76	71
query38	2856	2765	2716	2716
query39	840	805	814	805
query40	279	127	123	123
query41	50	47	54	47
query42	118	99	104	99
query43	608	532	544	532
query44	1204	729	745	729
query45	195	167	167	167
query46	1082	731	742	731
query47	1855	1750	1785	1750
query48	376	298	303	298
query49	1181	412	400	400
query50	773	385	385	385
query51	6736	6837	6719	6719
query52	115	98	99	98
query53	362	288	290	288
query54	1069	453	442	442
query55	78	77	79	77
query56	277	299	273	273
query57	1230	1060	1098	1060
query58	264	250	277	250
query59	3748	3426	3312	3312
query60	289	280	280	280
query61	90	93	90	90
query62	640	433	447	433
query63	315	287	291	287
query64	9921	2325	1715	1715
query65	3204	3111	3145	3111
query66	1242	340	337	337
query67	15372	14975	14868	14868
query68	5809	567	559	559
query69	555	451	391	391
query70	1192	1139	1063	1063
query71	431	272	278	272
query72	7524	5265	5344	5265
query73	775	328	331	328
query74	5891	5479	5544	5479
query75	3556	2669	2751	2669
query76	3678	915	964	915
query77	645	308	300	300
query78	10400	9844	9886	9844
query79	1833	526	528	526
query80	1216	466	462	462
query81	536	220	225	220
query82	707	111	118	111
query83	195	171	166	166
query84	272	90	86	86
query85	1438	290	326	290
query86	470	324	319	319
query87	3304	3079	3139	3079
query88	3869	2487	2432	2432
query89	475	400	377	377
query90	1819	197	195	195
query91	127	98	100	98
query92	67	49	51	49
query93	1750	518	507	507
query94	1175	198	189	189
query95	406	311	324	311
query96	588	269	277	269
query97	3249	3069	3053	3053
query98	224	201	193	193
query99	1196	873	833	833
Total cold run time: 274954 ms
Total hot run time: 173591 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

ClickBench: Total hot run time: 31.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 51398770f37e6becf4cf3cfdb66ab128c3f47402, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.06	0.06
query4	1.64	0.09	0.10
query5	0.52	0.50	0.51
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.54	0.49	0.48
query10	0.54	0.54	0.53
query11	0.16	0.12	0.12
query12	0.16	0.12	0.13
query13	0.60	0.58	0.61
query14	0.78	0.80	0.77
query15	0.83	0.83	0.82
query16	0.39	0.37	0.37
query17	1.06	0.99	0.98
query18	0.23	0.24	0.25
query19	1.81	1.77	1.72
query20	0.02	0.01	0.01
query21	15.43	0.66	0.65
query22	4.54	6.05	2.52
query23	18.33	1.38	1.30
query24	2.19	0.22	0.23
query25	0.15	0.09	0.09
query26	0.27	0.17	0.17
query27	0.07	0.07	0.09
query28	13.16	1.02	1.01
query29	12.62	3.36	3.38
query30	0.26	0.06	0.07
query31	2.90	0.39	0.39
query32	3.24	0.47	0.47
query33	2.92	2.93	2.89
query34	17.18	4.37	4.42
query35	4.49	4.53	4.46
query36	0.65	0.47	0.46
query37	0.18	0.16	0.16
query38	0.15	0.14	0.15
query39	0.05	0.04	0.04
query40	0.18	0.14	0.14
query41	0.09	0.04	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.99 s
Total hot run time: 31.34 s

@doris-robot
Copy link

doris-robot commented Jun 13, 2024

TPC-DS: Total hot run time: 173065 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 53b09dded645b522e0f95759b637a0ac8b6166d7, data reload: false

query1	918	380	377	377
query2	6458	2333	2420	2333
query3	6648	203	204	203
query4	19221	17432	17370	17370
query5	4153	471	480	471
query6	241	157	159	157
query7	4590	303	301	301
query8	315	273	281	273
query9	8678	2472	2434	2434
query10	612	291	273	273
query11	10491	9929	9986	9929
query12	138	86	84	84
query13	1645	367	368	367
query14	9559	7515	7437	7437
query15	236	192	191	191
query16	7769	276	264	264
query17	1798	538	522	522
query18	1763	274	266	266
query19	190	154	149	149
query20	89	81	84	81
query21	214	137	127	127
query22	4359	4139	4017	4017
query23	33723	32891	32894	32891
query24	12122	2804	2788	2788
query25	654	347	371	347
query26	1765	152	155	152
query27	3007	325	332	325
query28	7614	2070	2090	2070
query29	1100	622	618	618
query30	293	152	147	147
query31	953	732	756	732
query32	96	53	54	53
query33	760	286	270	270
query34	1019	480	467	467
query35	731	605	608	605
query36	1120	913	966	913
query37	295	76	73	73
query38	2847	2722	2731	2722
query39	910	809	807	807
query40	284	124	125	124
query41	54	52	50	50
query42	115	97	101	97
query43	560	534	552	534
query44	1223	725	738	725
query45	199	168	170	168
query46	1072	745	707	707
query47	1829	1770	1754	1754
query48	386	317	299	299
query49	1199	410	399	399
query50	763	396	382	382
query51	6754	6724	6666	6666
query52	110	93	91	91
query53	360	292	289	289
query54	988	442	449	442
query55	75	72	76	72
query56	287	255	260	255
query57	1161	1049	1080	1049
query58	254	228	249	228
query59	3453	3364	3348	3348
query60	290	265	265	265
query61	93	87	87	87
query62	667	444	454	444
query63	324	292	296	292
query64	9830	2254	1724	1724
query65	3212	3140	3110	3110
query66	1336	328	338	328
query67	15451	14930	15056	14930
query68	4501	547	543	543
query69	448	305	314	305
query70	1124	1160	1154	1154
query71	400	271	272	271
query72	7237	5697	5309	5309
query73	742	330	339	330
query74	6023	5514	5392	5392
query75	3516	2651	2675	2651
query76	2817	966	913	913
query77	454	310	298	298
query78	10124	9859	9669	9669
query79	2149	577	514	514
query80	1073	467	462	462
query81	585	221	225	221
query82	785	108	109	108
query83	235	169	171	169
query84	238	84	85	84
query85	2044	287	272	272
query86	490	319	318	318
query87	3266	3079	3180	3079
query88	4094	2466	2456	2456
query89	491	394	388	388
query90	1827	196	193	193
query91	127	95	96	95
query92	65	48	47	47
query93	2500	516	504	504
query94	1289	187	191	187
query95	407	316	322	316
query96	599	278	279	278
query97	3204	3055	3049	3049
query98	211	198	192	192
query99	1274	833	841	833
Total cold run time: 275762 ms
Total hot run time: 173065 ms

@doris-robot
Copy link

doris-robot commented Jun 13, 2024

ClickBench: Total hot run time: 30.89 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 53b09dded645b522e0f95759b637a0ac8b6166d7, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.24	0.06	0.06
query4	1.66	0.08	0.08
query5	0.50	0.47	0.48
query6	1.13	0.72	0.74
query7	0.03	0.02	0.01
query8	0.06	0.04	0.04
query9	0.54	0.48	0.48
query10	0.55	0.54	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.81	0.78
query15	0.84	0.81	0.81
query16	0.37	0.37	0.37
query17	1.04	1.03	1.03
query18	0.24	0.23	0.24
query19	1.76	1.69	1.67
query20	0.01	0.00	0.01
query21	15.43	0.66	0.65
query22	4.38	6.80	2.18
query23	18.23	1.47	1.31
query24	2.16	0.22	0.21
query25	0.15	0.08	0.08
query26	0.26	0.18	0.17
query27	0.09	0.08	0.09
query28	13.25	1.01	0.99
query29	12.65	3.29	3.29
query30	0.26	0.07	0.05
query31	2.88	0.39	0.38
query32	3.29	0.48	0.47
query33	2.90	2.95	2.92
query34	16.99	4.43	4.44
query35	4.56	4.46	4.58
query36	0.65	0.46	0.48
query37	0.18	0.15	0.15
query38	0.15	0.13	0.14
query39	0.05	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.64 s
Total hot run time: 30.89 s

@zddr
Copy link
Contributor Author

zddr commented Jun 17, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 17, 2024

TPC-H: Total hot run time: 39623 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2101467e8533d350584a0c4c2ad4528dac9efcd4, data reload: false

------ Round 1 ----------------------------------
q1	17619	4278	4292	4278
q2	2016	188	192	188
q3	10453	1151	1162	1151
q4	10199	776	749	749
q5	7480	2673	2642	2642
q6	218	136	135	135
q7	961	606	592	592
q8	9228	2062	2044	2044
q9	8931	6497	6458	6458
q10	8970	3680	3724	3680
q11	467	244	244	244
q12	481	234	225	225
q13	17817	2959	2986	2959
q14	269	216	219	216
q15	519	503	479	479
q16	507	377	381	377
q17	957	586	615	586
q18	7921	7334	7273	7273
q19	3574	1522	1502	1502
q20	651	309	324	309
q21	4882	3204	3214	3204
q22	387	337	332	332
Total cold run time: 114507 ms
Total hot run time: 39623 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4296	4205	4166	4166
q2	385	273	264	264
q3	2919	2732	2733	2732
q4	2039	1692	1730	1692
q5	5490	5637	5482	5482
q6	225	132	132	132
q7	2188	1836	1791	1791
q8	3318	3462	3395	3395
q9	8663	8675	8717	8675
q10	4113	3789	3932	3789
q11	571	496	490	490
q12	761	608	572	572
q13	17044	3197	3178	3178
q14	322	271	274	271
q15	530	485	492	485
q16	510	428	433	428
q17	1809	1492	1560	1492
q18	8080	7967	7810	7810
q19	2124	1627	1707	1627
q20	2110	1900	1894	1894
q21	7020	4937	4914	4914
q22	632	522	532	522
Total cold run time: 75149 ms
Total hot run time: 55801 ms

@doris-robot
Copy link

doris-robot commented Jun 17, 2024

TPC-DS: Total hot run time: 173639 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2101467e8533d350584a0c4c2ad4528dac9efcd4, data reload: false

query1	915	390	376	376
query2	6444	2402	2375	2375
query3	6641	203	206	203
query4	20192	17100	17077	17077
query5	3585	482	480	480
query6	237	154	157	154
query7	4579	298	293	293
query8	336	291	280	280
query9	8549	2377	2360	2360
query10	551	294	281	281
query11	10371	10006	9958	9958
query12	132	87	84	84
query13	1633	362	370	362
query14	9321	7143	7707	7143
query15	237	193	181	181
query16	7706	264	279	264
query17	1382	558	511	511
query18	1939	271	267	267
query19	206	153	158	153
query20	91	85	86	85
query21	206	133	124	124
query22	4184	3965	4000	3965
query23	33605	33886	33775	33775
query24	11248	2886	2936	2886
query25	587	377	363	363
query26	715	164	156	156
query27	2322	319	325	319
query28	5885	2125	2109	2109
query29	889	637	648	637
query30	259	154	156	154
query31	959	777	745	745
query32	98	55	52	52
query33	684	291	288	288
query34	901	478	485	478
query35	745	666	649	649
query36	1122	977	973	973
query37	141	76	74	74
query38	2971	2828	2823	2823
query39	881	848	879	848
query40	214	132	141	132
query41	55	55	49	49
query42	122	100	101	100
query43	561	556	559	556
query44	1105	719	727	719
query45	201	166	163	163
query46	1056	745	719	719
query47	1859	1781	1767	1767
query48	367	300	292	292
query49	845	409	412	409
query50	780	388	408	388
query51	6745	6688	6799	6688
query52	108	96	93	93
query53	364	291	284	284
query54	889	449	435	435
query55	74	74	74	74
query56	285	263	273	263
query57	1128	1054	1060	1054
query58	264	238	262	238
query59	3331	3256	3201	3201
query60	315	283	294	283
query61	128	113	111	111
query62	654	441	439	439
query63	313	285	285	285
query64	8500	2185	1744	1744
query65	3203	3090	3075	3075
query66	740	323	350	323
query67	15391	14808	14733	14733
query68	4528	542	541	541
query69	506	453	350	350
query70	1180	1149	1112	1112
query71	402	337	285	285
query72	7016	5953	5404	5404
query73	750	322	326	322
query74	5886	5460	5558	5460
query75	3439	2697	2702	2697
query76	2538	948	889	889
query77	435	312	302	302
query78	10327	9765	9811	9765
query79	2013	520	526	520
query80	2287	465	473	465
query81	596	225	225	225
query82	1116	102	102	102
query83	311	169	168	168
query84	266	90	84	84
query85	1165	288	276	276
query86	462	309	308	308
query87	3262	3096	3042	3042
query88	2982	2346	2346	2346
query89	456	375	382	375
query90	1672	194	192	192
query91	132	105	101	101
query92	58	50	49	49
query93	1920	512	512	512
query94	1081	194	189	189
query95	406	334	402	334
query96	580	272	262	262
query97	3220	3026	3036	3026
query98	216	206	194	194
query99	1177	820	844	820
Total cold run time: 265431 ms
Total hot run time: 173639 ms

@doris-robot
Copy link

doris-robot commented Jun 17, 2024

ClickBench: Total hot run time: 30.31 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2101467e8533d350584a0c4c2ad4528dac9efcd4, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.04
query4	1.68	0.08	0.09
query5	0.50	0.49	0.49
query6	1.13	0.73	0.72
query7	0.02	0.02	0.02
query8	0.06	0.04	0.04
query9	0.54	0.49	0.48
query10	0.56	0.56	0.54
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.79	0.79	0.77
query15	0.83	0.82	0.83
query16	0.34	0.36	0.36
query17	0.95	0.97	0.97
query18	0.23	0.22	0.26
query19	1.89	1.74	1.73
query20	0.01	0.01	0.01
query21	15.41	0.64	0.64
query22	4.44	7.55	1.71
query23	18.30	1.50	1.27
query24	2.15	0.23	0.23
query25	0.16	0.09	0.08
query26	0.27	0.17	0.17
query27	0.07	0.08	0.08
query28	13.17	1.04	1.00
query29	12.60	3.26	3.27
query30	0.26	0.07	0.06
query31	2.86	0.39	0.39
query32	3.27	0.47	0.49
query33	2.96	2.88	2.88
query34	16.96	4.40	4.39
query35	4.47	4.47	4.50
query36	0.66	0.48	0.46
query37	0.19	0.15	0.15
query38	0.15	0.15	0.14
query39	0.05	0.03	0.03
query40	0.17	0.13	0.14
query41	0.09	0.05	0.04
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.53 s
Total hot run time: 30.31 s

@zddr
Copy link
Contributor Author

zddr commented Jun 20, 2024

run buildall

1 similar comment
@zddr
Copy link
Contributor Author

zddr commented Jun 20, 2024

run buildall

@zddr
Copy link
Contributor Author

zddr commented Jun 20, 2024

run performance

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

TPC-H: Total hot run time: 40078 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f79ba438aedb5d462c52aa0285cb1a69a1b08e0a, data reload: false

------ Round 1 ----------------------------------
q1	17705	4435	4276	4276
q2	2044	190	195	190
q3	10452	1100	1128	1100
q4	10203	826	754	754
q5	7472	2719	2636	2636
q6	222	137	135	135
q7	991	605	592	592
q8	9229	2080	2101	2080
q9	8980	6508	6523	6508
q10	9046	3781	3741	3741
q11	448	245	243	243
q12	486	228	222	222
q13	17764	2968	3033	2968
q14	287	219	213	213
q15	520	485	488	485
q16	552	377	384	377
q17	974	674	673	673
q18	8262	7624	7528	7528
q19	4914	1523	1568	1523
q20	662	340	333	333
q21	4898	3165	3848	3165
q22	417	336	343	336
Total cold run time: 116528 ms
Total hot run time: 40078 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4260	4235	4235
q2	379	275	272	272
q3	3025	2981	2959	2959
q4	1995	1665	1723	1665
q5	5620	5522	5498	5498
q6	226	125	128	125
q7	2291	1880	1915	1880
q8	3350	3464	3428	3428
q9	8750	8811	8831	8811
q10	4172	3723	3802	3723
q11	615	503	514	503
q12	815	655	624	624
q13	16549	3187	3132	3132
q14	305	292	281	281
q15	552	484	478	478
q16	507	442	434	434
q17	1823	1508	1494	1494
q18	8227	8043	7900	7900
q19	2979	1651	1525	1525
q20	3075	1896	1883	1883
q21	5148	5006	4768	4768
q22	660	587	556	556
Total cold run time: 75455 ms
Total hot run time: 56174 ms

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

TPC-DS: Total hot run time: 175100 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f79ba438aedb5d462c52aa0285cb1a69a1b08e0a, data reload: false

query1	924	379	373	373
query2	6445	2416	2351	2351
query3	6633	214	215	214
query4	19263	17524	17413	17413
query5	3667	473	456	456
query6	241	155	163	155
query7	4574	291	288	288
query8	311	296	289	289
query9	8631	2433	2419	2419
query10	562	305	283	283
query11	10468	10030	10026	10026
query12	123	93	85	85
query13	1651	386	349	349
query14	9922	7631	7723	7631
query15	249	190	195	190
query16	7809	254	263	254
query17	1922	545	524	524
query18	1997	272	270	270
query19	187	156	154	154
query20	90	86	82	82
query21	200	131	126	126
query22	4355	4205	4029	4029
query23	34153	33687	33652	33652
query24	11012	2934	2849	2849
query25	570	380	363	363
query26	1009	156	154	154
query27	2362	334	319	319
query28	6912	2146	2107	2107
query29	859	640	612	612
query30	230	167	160	160
query31	964	771	757	757
query32	97	54	54	54
query33	754	283	296	283
query34	1055	478	489	478
query35	729	635	631	631
query36	1107	956	996	956
query37	142	74	76	74
query38	2982	2850	2821	2821
query39	868	838	838	838
query40	207	131	131	131
query41	55	52	49	49
query42	113	106	106	106
query43	613	532	519	519
query44	1219	722	749	722
query45	189	163	159	159
query46	1068	732	719	719
query47	1852	1782	1793	1782
query48	372	291	284	284
query49	826	393	403	393
query50	776	384	387	384
query51	6848	6788	6738	6738
query52	108	90	93	90
query53	358	287	283	283
query54	916	442	436	436
query55	77	71	72	71
query56	276	255	262	255
query57	1135	1070	1012	1012
query58	243	251	254	251
query59	3211	3283	3226	3226
query60	323	297	297	297
query61	122	110	107	107
query62	612	455	455	455
query63	323	296	289	289
query64	8593	2348	1848	1848
query65	3273	3157	3089	3089
query66	799	335	334	334
query67	15576	15091	15016	15016
query68	4606	523	521	521
query69	543	310	317	310
query70	1183	1148	1176	1148
query71	411	280	276	276
query72	8253	5568	5897	5568
query73	762	321	324	321
query74	5909	5536	5487	5487
query75	3838	2714	2678	2678
query76	2473	1000	896	896
query77	697	307	417	307
query78	10446	9849	9851	9849
query79	4922	536	528	528
query80	1557	484	478	478
query81	561	222	226	222
query82	1303	105	111	105
query83	367	173	171	171
query84	267	83	88	83
query85	1962	338	284	284
query86	477	333	321	321
query87	3315	3076	3067	3067
query88	4665	2361	2330	2330
query89	490	392	389	389
query90	1876	186	192	186
query91	132	99	99	99
query92	65	50	49	49
query93	5243	507	505	505
query94	1282	188	186	186
query95	404	311	311	311
query96	608	270	269	269
query97	3286	3040	3018	3018
query98	223	195	194	194
query99	1171	830	837	830
Total cold run time: 279409 ms
Total hot run time: 175100 ms

@doris-robot
Copy link

doris-robot commented Jun 20, 2024

ClickBench: Total hot run time: 30.49 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f79ba438aedb5d462c52aa0285cb1a69a1b08e0a, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.09
query5	0.49	0.50	0.49
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.05
query9	0.54	0.48	0.49
query10	0.55	0.54	0.54
query11	0.15	0.12	0.13
query12	0.15	0.12	0.13
query13	0.60	0.59	0.60
query14	0.78	0.78	0.78
query15	0.84	0.83	0.83
query16	0.36	0.36	0.38
query17	0.97	1.02	1.05
query18	0.21	0.26	0.23
query19	1.79	1.68	1.77
query20	0.01	0.01	0.01
query21	15.41	0.66	0.65
query22	3.50	8.40	1.58
query23	18.31	1.45	1.36
query24	2.10	0.23	0.23
query25	0.16	0.09	0.08
query26	0.28	0.19	0.18
query27	0.08	0.08	0.08
query28	13.24	1.03	1.01
query29	12.73	3.28	3.34
query30	0.26	0.06	0.06
query31	2.87	0.40	0.39
query32	3.27	0.48	0.48
query33	2.93	2.96	2.89
query34	17.04	4.44	4.46
query35	4.47	4.48	4.51
query36	0.66	0.46	0.46
query37	0.19	0.17	0.16
query38	0.15	0.15	0.14
query39	0.05	0.04	0.04
query40	0.18	0.14	0.14
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.74 s
Total hot run time: 30.49 s

@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2024

PR approved by anyone and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 21, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 21, 2024

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit 025b12d into apache:master Jun 21, 2024
iszhangpch pushed a commit to iszhangpch/doris-p that referenced this pull request Jun 21, 2024
@zddr
[enhance](mtmv)show create materialized view (apache#36188)
673aab7 
- support show create materialized view
- Prohibit show create table from specifying MTMV
  
 mysql> show create MATERIALIZED VIEW mv1\G
*************************** 1. row ***************************
Materialized View: mv1
Create Materialized View: CREATE MATERIALIZED VIEW mv1
(k1,k2)
BUILD DEFERRED REFRESH AUTO ON MANUAL
DUPLICATE KEY(`k1`, `k2`)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
)
AS SELECT * from user1
1 row in set (0.01 sec)

mysql> show create table rollup_mv;
ERROR 1105 (HY000): errCode = 2, detailMessage = not support async
materialized view, please use `show create materialized view`
dataroaring pushed a commit that referenced this pull request Jun 21, 2024
@zddr @dataroaring
[enhance](mtmv)show create materialized view (#36188)
daf2fb6 
- support show create materialized view
- Prohibit show create table from specifying MTMV
  
 mysql> show create MATERIALIZED VIEW mv1\G
*************************** 1. row ***************************
Materialized View: mv1
Create Materialized View: CREATE MATERIALIZED VIEW mv1
(k1,k2)
BUILD DEFERRED REFRESH AUTO ON MANUAL
DUPLICATE KEY(`k1`, `k2`)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
)
AS SELECT * from user1
1 row in set (0.01 sec)

mysql> show create table rollup_mv;
ERROR 1105 (HY000): errCode = 2, detailMessage = not support async
materialized view, please use `show create materialized view`
zddr added a commit to zddr/incubator-doris that referenced this pull request Jul 2, 2024
@zddr
[enhance](mtmv)show create materialized view (apache#36188)
8f19362 
- support show create materialized view
- Prohibit show create table from specifying MTMV

 mysql> show create MATERIALIZED VIEW mv1\G
*************************** 1. row ***************************
Materialized View: mv1
Create Materialized View: CREATE MATERIALIZED VIEW mv1
(k1,k2)
BUILD DEFERRED REFRESH AUTO ON MANUAL
DUPLICATE KEY(`k1`, `k2`)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
)
AS SELECT * from user1
1 row in set (0.01 sec)

mysql> show create table rollup_mv;
ERROR 1105 (HY000): errCode = 2, detailMessage = not support async
materialized view, please use `show create materialized view`
@zddr zddr mentioned this pull request Jul 2, 2024
Merged
morningman pushed a commit that referenced this pull request Jul 3, 2024
@zddr
[enhance](mtmv)show create materialized view (#36188) (#37125)
077fda4 
pick: #36188
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

+1 more reviewer

@seawinde seawinde seawinde approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@zddr @doris-robot @morrySnow @seawinde @morningman @dataroaring