Skip to content

[fix](mtmv)transfer col in mysql varchar to text when create MTMV #37668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 18 commits into from
Aug 21, 2024
Merged

[fix](mtmv)transfer col in mysql varchar to text when create MTMV #37668

morrySnow merged 18 commits into from
Aug 21, 2024

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented Jul 11, 2024 •
edited
Loading

varchar(100) in mysql represents the ability to store 100 Chinese characters

but in doris， may need varchar(400)

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr
Copy link
Contributor Author

zddr commented Jul 11, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-H: Total hot run time: 40123 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 94a3d93411745d14b0bd6033e7665eb1b5bb332f, data reload: false

------ Round 1 ----------------------------------
q1	17620	4367	4265	4265
q2	2032	194	192	192
q3	10427	1276	1140	1140
q4	10193	919	781	781
q5	7956	2697	2648	2648
q6	229	143	142	142
q7	999	627	627	627
q8	9483	2083	2068	2068
q9	8806	6551	6538	6538
q10	8762	3855	3728	3728
q11	458	246	239	239
q12	395	243	239	239
q13	18656	2978	2994	2978
q14	284	248	236	236
q15	526	486	482	482
q16	503	385	380	380
q17	972	629	732	629
q18	8167	7439	7445	7439
q19	2083	1459	1534	1459
q20	685	330	351	330
q21	4945	3260	3240	3240
q22	387	353	343	343
Total cold run time: 114568 ms
Total hot run time: 40123 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4383	4340	4245	4245
q2	378	279	264	264
q3	3036	2815	2765	2765
q4	1872	1633	1610	1610
q5	5292	5262	5291	5262
q6	219	131	133	131
q7	2087	1767	1665	1665
q8	3202	3335	3293	3293
q9	8380	8382	8390	8382
q10	3888	3674	3707	3674
q11	562	502	493	493
q12	777	647	642	642
q13	17213	3007	3003	3003
q14	299	271	272	271
q15	526	478	478	478
q16	472	417	430	417
q17	1768	1508	1460	1460
q18	7566	7732	7422	7422
q19	2938	1611	1543	1543
q20	2031	1789	1765	1765
q21	4788	4800	4756	4756
q22	621	561	550	550
Total cold run time: 72298 ms
Total hot run time: 54091 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-DS: Total hot run time: 173716 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 94a3d93411745d14b0bd6033e7665eb1b5bb332f, data reload: false

query1	931	376	368	368
query2	6454	2392	2298	2298
query3	6653	214	226	214
query4	28264	17548	17228	17228
query5	4185	505	488	488
query6	279	183	157	157
query7	4596	301	300	300
query8	342	302	318	302
query9	8690	2497	2476	2476
query10	442	303	273	273
query11	11342	9997	9990	9990
query12	129	86	81	81
query13	1635	363	366	363
query14	10291	7710	8082	7710
query15	239	184	184	184
query16	7224	317	314	314
query17	1798	568	528	528
query18	1824	282	276	276
query19	197	146	153	146
query20	87	82	84	82
query21	204	126	131	126
query22	4286	4179	3890	3890
query23	33799	34153	33182	33182
query24	11764	2888	2835	2835
query25	500	362	375	362
query26	1187	151	156	151
query27	2548	274	278	274
query28	7113	2093	2078	2078
query29	860	651	644	644
query30	288	151	153	151
query31	980	739	801	739
query32	98	53	60	53
query33	770	302	290	290
query34	953	491	492	491
query35	664	558	576	558
query36	1054	964	949	949
query37	133	81	83	81
query38	2889	2748	2766	2748
query39	885	816	796	796
query40	281	119	116	116
query41	55	49	51	49
query42	116	96	104	96
query43	576	521	549	521
query44	1226	735	725	725
query45	193	168	165	165
query46	1091	729	731	729
query47	1867	1816	1798	1798
query48	377	306	308	306
query49	1164	412	400	400
query50	781	408	401	401
query51	6893	6787	6740	6740
query52	102	89	97	89
query53	362	297	303	297
query54	911	482	449	449
query55	75	76	78	76
query56	285	353	267	267
query57	1149	1025	1070	1025
query58	244	240	244	240
query59	3380	3058	3045	3045
query60	298	271	280	271
query61	96	94	93	93
query62	820	658	650	650
query63	327	293	287	287
query64	10408	2310	1998	1998
query65	3181	3144	3118	3118
query66	1254	330	328	328
query67	15481	14915	14743	14743
query68	4591	536	549	536
query69	549	416	335	335
query70	1205	1160	1168	1160
query71	444	281	284	281
query72	7203	5908	5326	5326
query73	757	332	331	331
query74	6068	5463	5642	5463
query75	3360	2725	2674	2674
query76	2800	982	857	857
query77	567	302	310	302
query78	10747	10104	9002	9002
query79	2940	527	540	527
query80	1324	482	533	482
query81	550	221	220	220
query82	1260	143	131	131
query83	240	171	167	167
query84	273	93	85	85
query85	1604	315	298	298
query86	478	303	305	303
query87	3274	3121	3079	3079
query88	4287	2453	2461	2453
query89	493	390	377	377
query90	1894	195	197	195
query91	133	106	105	105
query92	60	52	52	52
query93	3308	515	509	509
query94	1220	218	216	216
query95	408	323	323	323
query96	614	278	270	270
query97	3196	3010	3046	3010
query98	233	194	202	194
query99	1635	1255	1256	1255
Total cold run time: 287285 ms
Total hot run time: 173716 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

ClickBench: Total hot run time: 30.8 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 94a3d93411745d14b0bd6033e7665eb1b5bb332f, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.04	0.04
query3	0.21	0.05	0.06
query4	1.66	0.08	0.10
query5	0.49	0.47	0.50
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.56	0.47	0.49
query10	0.55	0.55	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.14
query13	0.60	0.58	0.59
query14	0.78	0.77	0.78
query15	0.85	0.80	0.81
query16	0.36	0.36	0.36
query17	0.98	0.93	0.98
query18	0.23	0.24	0.23
query19	1.83	1.70	1.71
query20	0.02	0.01	0.03
query21	15.40	0.75	0.64
query22	4.50	6.74	2.15
query23	18.30	1.29	1.24
query24	2.08	0.23	0.22
query25	0.16	0.09	0.08
query26	0.30	0.22	0.22
query27	0.45	0.23	0.23
query28	13.23	1.01	0.99
query29	12.64	3.33	3.28
query30	0.25	0.06	0.05
query31	2.87	0.39	0.40
query32	3.26	0.48	0.48
query33	2.90	2.94	2.96
query34	17.06	4.34	4.37
query35	4.45	4.42	4.49
query36	0.66	0.46	0.49
query37	0.19	0.16	0.15
query38	0.16	0.16	0.16
query39	0.05	0.03	0.04
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.05	0.05
Total cold run time: 110 s
Total hot run time: 30.8 s

@zddr
Copy link
Contributor Author

zddr commented Jul 11, 2024

run buildall

@zddr zddr requested a review from morrySnow July 11, 2024 11:26
@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-H: Total hot run time: 40151 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5b303a454b12089a9df12e6f1cd37e2f4bb8c00f, data reload: false

------ Round 1 ----------------------------------
q1	17595	4483	4276	4276
q2	2008	192	188	188
q3	10472	1226	1173	1173
q4	10191	819	827	819
q5	7790	2728	2658	2658
q6	227	137	140	137
q7	975	612	611	611
q8	9647	2102	2128	2102
q9	8935	6522	6598	6522
q10	8656	3808	3778	3778
q11	464	243	247	243
q12	402	230	230	230
q13	17777	3000	2989	2989
q14	269	231	238	231
q15	530	493	487	487
q16	505	389	375	375
q17	969	608	656	608
q18	8109	7518	7369	7369
q19	2233	1492	1562	1492
q20	688	325	323	323
q21	5021	3203	3206	3203
q22	386	339	337	337
Total cold run time: 113849 ms
Total hot run time: 40151 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4358	4317	4232	4232
q2	372	268	267	267
q3	3029	2718	2742	2718
q4	1849	1609	1612	1609
q5	5281	5291	5288	5288
q6	225	128	134	128
q7	2080	1722	1737	1722
q8	3174	3318	3297	3297
q9	8378	8407	8341	8341
q10	3875	3656	3692	3656
q11	576	483	478	478
q12	777	610	592	592
q13	16521	3012	3019	3012
q14	309	276	266	266
q15	522	481	476	476
q16	471	403	413	403
q17	1786	1477	1457	1457
q18	7550	7445	7252	7252
q19	3737	1721	1711	1711
q20	2006	1820	1778	1778
q21	4843	4629	4646	4629
q22	570	551	557	551
Total cold run time: 72289 ms
Total hot run time: 53863 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

TPC-DS: Total hot run time: 175066 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5b303a454b12089a9df12e6f1cd37e2f4bb8c00f, data reload: false

query1	935	389	371	371
query2	6458	2425	2385	2385
query3	6661	213	223	213
query4	28712	17795	17333	17333
query5	4174	480	486	480
query6	278	182	170	170
query7	4584	294	299	294
query8	317	292	295	292
query9	8590	2475	2473	2473
query10	437	290	272	272
query11	10610	9830	10284	9830
query12	139	85	84	84
query13	1642	382	375	375
query14	10292	8058	8035	8035
query15	240	187	188	187
query16	7561	318	328	318
query17	1720	556	522	522
query18	1723	275	275	275
query19	198	147	146	146
query20	89	81	83	81
query21	211	133	125	125
query22	4299	4199	3987	3987
query23	33765	32845	33136	32845
query24	12070	2873	2802	2802
query25	635	368	373	368
query26	1696	152	154	152
query27	2788	273	272	272
query28	7572	2097	2096	2096
query29	964	651	641	641
query30	276	148	150	148
query31	1003	763	738	738
query32	93	53	54	53
query33	747	349	295	295
query34	906	490	491	490
query35	676	612	593	593
query36	1104	942	940	940
query37	271	84	85	84
query38	2854	2759	2729	2729
query39	857	797	789	789
query40	282	121	119	119
query41	55	51	54	51
query42	117	104	100	100
query43	598	542	530	530
query44	1249	748	714	714
query45	197	165	161	161
query46	1088	771	738	738
query47	1854	1777	1766	1766
query48	369	305	295	295
query49	1222	415	446	415
query50	786	403	409	403
query51	6833	6795	6795	6795
query52	114	91	94	91
query53	365	304	294	294
query54	1080	464	456	456
query55	77	74	76	74
query56	302	272	283	272
query57	1190	1051	1051	1051
query58	241	250	250	250
query59	3288	3243	3063	3063
query60	317	288	291	288
query61	98	101	98	98
query62	841	631	648	631
query63	332	296	299	296
query64	10511	2252	1633	1633
query65	3259	3105	3112	3105
query66	1354	335	336	335
query67	15500	15040	15021	15021
query68	4930	542	545	542
query69	645	482	348	348
query70	1116	1105	1075	1075
query71	444	289	287	287
query72	7411	5662	5927	5662
query73	768	336	330	330
query74	5887	5444	5483	5444
query75	3459	2696	2673	2673
query76	3090	903	863	863
query77	642	309	314	309
query78	9598	9819	9988	9819
query79	2903	534	532	532
query80	1924	473	487	473
query81	575	217	217	217
query82	775	140	139	139
query83	334	169	175	169
query84	277	90	90	90
query85	1604	314	311	311
query86	351	285	305	285
query87	3285	3137	3147	3137
query88	4235	2475	2460	2460
query89	492	386	377	377
query90	1418	193	196	193
query91	137	105	106	105
query92	61	49	50	49
query93	3773	515	510	510
query94	593	221	219	219
query95	401	337	332	332
query96	613	279	280	279
query97	3216	3048	3035	3035
query98	214	194	192	192
query99	1733	1282	1370	1282
Total cold run time: 288570 ms
Total hot run time: 175066 ms

@doris-robot
Copy link

doris-robot commented Jul 11, 2024

ClickBench: Total hot run time: 30.76 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5b303a454b12089a9df12e6f1cd37e2f4bb8c00f, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.04
query3	0.22	0.06	0.05
query4	1.67	0.07	0.07
query5	0.50	0.48	0.47
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.48	0.49
query10	0.56	0.54	0.55
query11	0.16	0.12	0.11
query12	0.15	0.13	0.13
query13	0.59	0.59	0.60
query14	0.75	0.77	0.75
query15	0.84	0.82	0.81
query16	0.37	0.37	0.35
query17	0.97	0.95	0.97
query18	0.23	0.22	0.22
query19	1.77	1.71	1.71
query20	0.01	0.01	0.02
query21	15.41	0.73	0.64
query22	4.80	6.51	2.14
query23	18.32	1.41	1.25
query24	2.12	0.22	0.23
query25	0.16	0.09	0.08
query26	0.30	0.22	0.22
query27	0.45	0.24	0.23
query28	13.22	1.02	1.00
query29	12.64	3.30	3.30
query30	0.25	0.06	0.05
query31	2.87	0.39	0.39
query32	3.27	0.48	0.47
query33	2.86	2.97	2.89
query34	16.91	4.36	4.29
query35	4.40	4.44	4.46
query36	0.66	0.49	0.47
query37	0.18	0.16	0.16
query38	0.16	0.16	0.15
query39	0.05	0.04	0.03
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.05	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.05 s
Total hot run time: 30.76 s

@zddr
Copy link
Contributor Author

zddr commented Jul 12, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jul 12, 2024

TPC-H: Total hot run time: 39970 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 489c003beb3c42ca56c9ef3b65e5b3a77b03a952, data reload: false

------ Round 1 ----------------------------------
q1	17926	4417	4367	4367
q2	2337	190	190	190
q3	10459	1225	1041	1041
q4	10189	884	818	818
q5	7528	2693	2669	2669
q6	218	139	137	137
q7	959	592	609	592
q8	9208	2068	2091	2068
q9	8606	6524	6550	6524
q10	8715	3754	3744	3744
q11	468	243	238	238
q12	397	236	223	223
q13	17751	2968	2945	2945
q14	285	228	228	228
q15	511	489	484	484
q16	471	392	375	375
q17	958	647	751	647
q18	8152	7448	7419	7419
q19	7992	1544	1465	1465
q20	701	319	323	319
q21	4824	3274	3141	3141
q22	396	345	336	336
Total cold run time: 119051 ms
Total hot run time: 39970 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4360	4228	4277	4228
q2	383	253	266	253
q3	3035	2726	2714	2714
q4	1879	1651	1548	1548
q5	5290	5291	5271	5271
q6	218	127	129	127
q7	2100	1694	1693	1693
q8	3175	3333	3302	3302
q9	8366	8370	8331	8331
q10	3957	3677	3702	3677
q11	614	478	485	478
q12	792	621	623	621
q13	17257	2965	2977	2965
q14	297	269	279	269
q15	525	486	468	468
q16	467	404	431	404
q17	1787	1465	1473	1465
q18	7678	7412	7397	7397
q19	4084	1624	1625	1624
q20	2018	1793	1765	1765
q21	4850	4664	4692	4664
q22	613	546	546	546
Total cold run time: 73745 ms
Total hot run time: 53810 ms

@doris-robot
Copy link

doris-robot commented Jul 12, 2024

TPC-DS: Total hot run time: 173551 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 489c003beb3c42ca56c9ef3b65e5b3a77b03a952, data reload: false

query1	923	374	360	360
query2	6454	2361	2431	2361
query3	6658	212	220	212
query4	28373	17524	17249	17249
query5	4233	471	464	464
query6	287	165	154	154
query7	4595	293	284	284
query8	315	299	304	299
query9	8504	2461	2440	2440
query10	446	302	273	273
query11	10554	9969	10065	9969
query12	127	85	81	81
query13	1654	372	380	372
query14	10051	7894	7589	7589
query15	235	190	192	190
query16	7902	326	326	326
query17	1772	582	534	534
query18	1871	283	277	277
query19	201	152	151	151
query20	95	85	85	85
query21	208	128	128	128
query22	4221	4032	3939	3939
query23	34064	33024	33643	33024
query24	12127	2803	2887	2803
query25	676	394	400	394
query26	1752	151	152	151
query27	2894	276	277	276
query28	7666	2110	2115	2110
query29	1079	643	622	622
query30	288	149	147	147
query31	960	745	761	745
query32	101	53	55	53
query33	781	307	303	303
query34	960	474	483	474
query35	701	564	575	564
query36	1082	927	947	927
query37	278	80	83	80
query38	2806	2774	2776	2774
query39	866	801	804	801
query40	289	123	120	120
query41	55	52	52	52
query42	117	101	106	101
query43	594	527	557	527
query44	1251	748	732	732
query45	191	166	167	166
query46	1080	742	701	701
query47	1839	1767	1771	1767
query48	386	303	298	298
query49	1159	425	424	424
query50	785	403	410	403
query51	6905	6664	6661	6661
query52	112	93	106	93
query53	361	300	300	300
query54	893	457	467	457
query55	80	75	79	75
query56	304	301	289	289
query57	1156	1034	1048	1034
query58	253	267	252	252
query59	3529	3211	3212	3211
query60	303	281	281	281
query61	98	118	97	97
query62	815	645	670	645
query63	320	286	285	285
query64	10485	2293	1683	1683
query65	3341	3081	3072	3072
query66	1206	333	326	326
query67	15409	15004	14866	14866
query68	6829	570	564	564
query69	772	455	370	370
query70	1183	1151	1109	1109
query71	519	278	279	278
query72	8881	5892	5345	5345
query73	819	333	333	333
query74	5870	5431	5482	5431
query75	4574	2689	2673	2673
query76	4867	887	893	887
query77	750	307	314	307
query78	9723	9072	9099	9072
query79	7333	535	524	524
query80	1078	489	480	480
query81	577	215	221	215
query82	794	140	139	139
query83	331	166	168	166
query84	281	87	91	87
query85	1516	297	333	297
query86	444	288	292	288
query87	3327	3109	3136	3109
query88	4626	2480	2479	2479
query89	505	381	374	374
query90	2094	192	199	192
query91	134	102	104	102
query92	66	48	48	48
query93	5733	525	518	518
query94	1491	218	210	210
query95	399	325	323	323
query96	620	276	275	275
query97	3184	3001	2997	2997
query98	209	201	199	199
query99	1517	1263	1257	1257
Total cold run time: 303122 ms
Total hot run time: 173551 ms

@doris-robot
Copy link

doris-robot commented Jul 12, 2024

ClickBench: Total hot run time: 30.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 489c003beb3c42ca56c9ef3b65e5b3a77b03a952, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.22	0.04	0.04
query4	1.68	0.07	0.07
query5	0.48	0.49	0.49
query6	1.15	0.71	0.72
query7	0.02	0.02	0.01
query8	0.06	0.05	0.04
query9	0.55	0.49	0.48
query10	0.54	0.54	0.55
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.75	0.79	0.80
query15	0.85	0.80	0.81
query16	0.35	0.35	0.37
query17	0.97	1.00	1.00
query18	0.22	0.23	0.22
query19	1.83	1.73	1.72
query20	0.01	0.00	0.01
query21	15.40	0.78	0.65
query22	4.43	7.03	1.87
query23	18.30	1.27	1.20
query24	2.20	0.24	0.22
query25	0.15	0.08	0.09
query26	0.30	0.21	0.21
query27	0.45	0.22	0.23
query28	13.26	1.00	0.99
query29	12.69	3.28	3.25
query30	0.25	0.06	0.05
query31	2.87	0.38	0.39
query32	3.29	0.46	0.45
query33	2.89	2.84	2.95
query34	16.79	4.35	4.36
query35	4.34	4.37	4.39
query36	0.64	0.48	0.49
query37	0.19	0.15	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.05
query42	0.07	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.68 s
Total hot run time: 30.34 s

@zddr
Copy link
Contributor Author

zddr commented Jul 12, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jul 12, 2024

TPC-H: Total hot run time: 39699 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2c9d531a08ed9016014a97853e2484832e5eff7b, data reload: false

------ Round 1 ----------------------------------
q1	17614	4329	4253	4253
q2	2021	196	184	184
q3	10461	1156	1158	1156
q4	10215	845	730	730
q5	7537	2653	2657	2653
q6	223	139	137	137
q7	950	615	596	596
q8	9220	2080	2054	2054
q9	8891	6557	6550	6550
q10	8829	3729	3778	3729
q11	452	232	236	232
q12	470	229	228	228
q13	17782	2970	2988	2970
q14	282	231	236	231
q15	515	474	487	474
q16	504	385	373	373
q17	955	676	676	676
q18	8035	7492	7347	7347
q19	5310	1478	1361	1361
q20	699	320	329	320
q21	4842	3226	3164	3164
q22	350	281	296	281
Total cold run time: 116157 ms
Total hot run time: 39699 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4253	4290	4253
q2	363	261	261	261
q3	3020	2748	2898	2748
q4	1950	1726	1660	1660
q5	5628	5481	5536	5481
q6	229	157	131	131
q7	2148	1799	1874	1799
q8	3248	3409	3357	3357
q9	8808	8795	8844	8795
q10	4070	3926	3825	3825
q11	585	477	499	477
q12	821	641	663	641
q13	16936	3201	3185	3185
q14	330	284	293	284
q15	560	503	476	476
q16	484	430	432	430
q17	1810	1506	1502	1502
q18	8179	7988	7825	7825
q19	1736	1504	1549	1504
q20	2187	1891	1856	1856
q21	5129	4888	4765	4765
q22	558	529	471	471
Total cold run time: 73171 ms
Total hot run time: 55726 ms

@doris-robot
Copy link

doris-robot commented Jul 12, 2024

TPC-DS: Total hot run time: 173273 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2c9d531a08ed9016014a97853e2484832e5eff7b, data reload: false

query1	910	368	373	368
query2	6444	1902	1802	1802
query3	6636	203	214	203
query4	28097	17493	17491	17491
query5	3645	498	482	482
query6	264	158	162	158
query7	4584	299	289	289
query8	233	188	186	186
query9	8474	2437	2407	2407
query10	427	288	270	270
query11	11353	9921	10029	9921
query12	114	84	86	84
query13	1644	369	376	369
query14	10009	7732	7729	7729
query15	231	166	168	166
query16	7311	309	316	309
query17	1353	546	528	528
query18	1750	271	279	271
query19	196	147	148	147
query20	90	79	81	79
query21	206	129	121	121
query22	4360	4040	3983	3983
query23	33897	33687	33495	33495
query24	11152	2976	2927	2927
query25	616	382	425	382
query26	706	155	155	155
query27	2306	279	284	279
query28	6475	2114	2093	2093
query29	886	656	636	636
query30	262	162	151	151
query31	998	766	783	766
query32	101	55	55	55
query33	753	312	290	290
query34	1025	502	513	502
query35	724	620	601	601
query36	1149	1000	984	984
query37	157	93	92	92
query38	2937	2883	2854	2854
query39	897	866	833	833
query40	221	128	128	128
query41	53	48	50	48
query42	123	110	104	104
query43	533	486	463	463
query44	1307	755	752	752
query45	197	169	162	162
query46	1111	767	724	724
query47	1827	1746	1775	1746
query48	377	301	306	301
query49	848	433	437	433
query50	784	401	398	398
query51	7010	6838	6791	6791
query52	109	96	98	96
query53	354	303	306	303
query54	884	465	462	462
query55	79	76	76	76
query56	307	287	282	282
query57	1147	1080	1045	1045
query58	252	258	286	258
query59	2830	2636	2537	2537
query60	308	300	292	292
query61	118	116	117	116
query62	818	641	633	633
query63	328	303	302	302
query64	9577	2208	1634	1634
query65	3174	3091	3092	3091
query66	771	325	326	325
query67	15413	14887	14858	14858
query68	8310	569	614	569
query69	702	438	354	354
query70	1170	1129	1128	1128
query71	495	281	289	281
query72	6979	5591	5281	5281
query73	1191	324	330	324
query74	5911	5472	5536	5472
query75	4039	2687	2724	2687
query76	4213	1002	903	903
query77	677	309	318	309
query78	9658	8964	8915	8915
query79	7645	530	538	530
query80	2226	487	475	475
query81	590	224	225	224
query82	297	129	134	129
query83	287	171	168	168
query84	278	87	87	87
query85	878	368	304	304
query86	356	297	332	297
query87	3273	3074	3101	3074
query88	4871	2467	2494	2467
query89	526	396	384	384
query90	1882	198	206	198
query91	129	99	99	99
query92	65	52	50	50
query93	5911	522	518	518
query94	1233	217	215	215
query95	405	317	331	317
query96	620	282	271	271
query97	3194	2996	3036	2996
query98	221	199	197	197
query99	1557	1232	1282	1232
Total cold run time: 294021 ms
Total hot run time: 173273 ms

@doris-robot
Copy link

doris-robot commented Jul 12, 2024

ClickBench: Total hot run time: 30.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2c9d531a08ed9016014a97853e2484832e5eff7b, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.68	0.08	0.07
query5	0.49	0.47	0.48
query6	1.15	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.56	0.48	0.50
query10	0.55	0.55	0.56
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.60	0.58
query14	0.78	0.78	0.82
query15	0.84	0.82	0.82
query16	0.35	0.37	0.37
query17	0.95	1.02	1.04
query18	0.23	0.22	0.22
query19	1.74	1.66	1.76
query20	0.01	0.01	0.00
query21	15.40	0.74	0.65
query22	4.33	6.92	2.11
query23	18.27	1.38	1.24
query24	2.25	0.21	0.22
query25	0.15	0.08	0.09
query26	0.29	0.21	0.21
query27	0.45	0.24	0.23
query28	13.16	1.03	0.99
query29	12.63	3.37	3.34
query30	0.25	0.06	0.06
query31	2.85	0.39	0.38
query32	3.28	0.48	0.47
query33	2.90	2.88	2.90
query34	17.38	4.34	4.37
query35	4.43	4.40	4.40
query36	0.65	0.46	0.47
query37	0.19	0.15	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.04
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.05 s
Total hot run time: 30.77 s

@zddr
Copy link
Contributor Author

zddr commented Jul 19, 2024

run buildall

@zddr
Copy link
Contributor Author

zddr commented Aug 19, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Aug 19, 2024

TPC-H: Total hot run time: 38542 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1298f2c6808f723bb5ce43b45751236febcc2b6e, data reload: false

------ Round 1 ----------------------------------
q1	18016	4510	4456	4456
q2	2504	242	232	232
q3	11964	1143	1116	1116
q4	10985	723	820	723
q5	8335	2881	2804	2804
q6	266	158	160	158
q7	1017	677	653	653
q8	9391	2108	2074	2074
q9	7280	6559	6551	6551
q10	7069	2224	2276	2224
q11	566	279	283	279
q12	427	257	268	257
q13	18945	3046	3020	3020
q14	315	268	260	260
q15	561	532	535	532
q16	537	425	416	416
q17	999	678	720	678
q18	7422	6844	6876	6844
q19	7168	1093	1089	1089
q20	744	362	363	362
q21	3924	3069	2775	2775
q22	1147	1049	1039	1039
Total cold run time: 119582 ms
Total hot run time: 38542 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4555	4326	4318	4318
q2	431	315	320	315
q3	2884	2666	2632	2632
q4	1898	1643	1622	1622
q5	5390	5400	5380	5380
q6	247	154	155	154
q7	2112	1726	1724	1724
q8	3236	3383	3344	3344
q9	8445	8491	8429	8429
q10	3442	3189	3199	3189
q11	630	535	533	533
q12	833	614	633	614
q13	17336	2985	2973	2973
q14	332	298	289	289
q15	550	516	507	507
q16	510	457	452	452
q17	1805	1505	1487	1487
q18	7847	7474	7362	7362
q19	6313	1616	1433	1433
q20	2055	1837	1822	1822
q21	13815	5182	4983	4983
q22	1120	1025	1007	1007
Total cold run time: 85786 ms
Total hot run time: 54569 ms

@doris-robot
Copy link

doris-robot commented Aug 19, 2024

TPC-DS: Total hot run time: 191678 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1298f2c6808f723bb5ce43b45751236febcc2b6e, data reload: false

query1	972	403	393	393
query2	6773	2025	1874	1874
query3	6660	227	250	227
query4	27788	23185	23303	23185
query5	4399	737	734	734
query6	310	220	215	215
query7	4617	340	326	326
query8	500	433	444	433
query9	8555	2526	2503	2503
query10	528	355	338	338
query11	16111	15103	15126	15103
query12	194	143	140	140
query13	1711	471	460	460
query14	9749	7779	7272	7272
query15	254	194	203	194
query16	7669	494	468	468
query17	1654	618	606	606
query18	1788	349	339	339
query19	222	176	171	171
query20	146	140	138	138
query21	255	149	145	145
query22	4321	4172	4020	4020
query23	34109	33368	33182	33182
query24	11578	3017	2929	2929
query25	650	433	425	425
query26	1202	186	181	181
query27	2735	303	302	302
query28	7084	2140	2099	2099
query29	847	459	441	441
query30	337	187	180	180
query31	1070	820	841	820
query32	122	81	82	81
query33	812	351	351	351
query34	903	510	515	510
query35	910	788	768	768
query36	1165	955	1012	955
query37	176	106	106	106
query38	4032	3872	3798	3798
query39	1518	1469	1476	1469
query40	248	162	159	159
query41	148	155	139	139
query42	138	119	121	119
query43	540	499	511	499
query44	1237	788	804	788
query45	225	195	197	195
query46	1133	799	763	763
query47	1920	1822	1841	1822
query48	434	344	338	338
query49	1264	616	604	604
query50	884	489	477	477
query51	6792	6904	6791	6791
query52	123	111	115	111
query53	311	233	229	229
query54	954	513	511	511
query55	93	88	90	88
query56	338	322	315	315
query57	1200	1116	1140	1116
query58	327	309	324	309
query59	2942	2816	2836	2816
query60	362	334	345	334
query61	154	148	147	147
query62	891	698	716	698
query63	271	240	236	236
query64	5365	2405	1869	1869
query65	3232	3184	3225	3184
query66	1125	687	679	679
query67	15331	14709	14848	14709
query68	6172	586	583	583
query69	752	414	348	348
query70	1247	1173	1134	1134
query71	560	326	314	314
query72	7571	2323	2135	2135
query73	823	355	355	355
query74	9019	8825	8867	8825
query75	4748	2749	2745	2745
query76	4154	1082	996	996
query77	844	451	443	443
query78	9744	10775	9725	9725
query79	10296	562	554	554
query80	1040	621	614	614
query81	614	258	261	258
query82	378	161	162	161
query83	407	215	222	215
query84	298	103	100	100
query85	772	432	354	354
query86	483	333	331	331
query87	4406	4366	4232	4232
query88	4328	2574	2464	2464
query89	448	336	334	334
query90	2101	245	242	242
query91	161	133	131	131
query92	96	76	77	76
query93	1983	560	560	560
query94	1039	321	311	311
query95	395	302	300	300
query96	621	284	287	284
query97	3285	3090	3088	3088
query98	250	227	238	227
query99	1563	1320	1313	1313
Total cold run time: 303006 ms
Total hot run time: 191678 ms

@doris-robot
Copy link

doris-robot commented Aug 19, 2024

ClickBench: Total hot run time: 31.61 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1298f2c6808f723bb5ce43b45751236febcc2b6e, data reload: false

query1	0.04	0.04	0.05
query2	0.09	0.04	0.04
query3	0.22	0.06	0.05
query4	1.66	0.08	0.08
query5	0.51	0.49	0.48
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.04
query9	0.54	0.50	0.49
query10	0.57	0.54	0.54
query11	0.16	0.13	0.13
query12	0.16	0.13	0.13
query13	0.63	0.60	0.60
query14	0.78	0.79	0.78
query15	0.85	0.82	0.82
query16	0.36	0.38	0.36
query17	0.97	1.06	1.04
query18	0.21	0.20	0.22
query19	1.88	1.68	1.70
query20	0.01	0.01	0.02
query21	15.39	0.69	0.68
query22	4.19	6.75	2.50
query23	18.26	1.46	1.28
query24	2.09	0.22	0.22
query25	0.16	0.10	0.09
query26	0.29	0.18	0.19
query27	0.09	0.07	0.08
query28	13.34	1.03	1.01
query29	12.66	3.36	3.30
query30	0.43	0.25	0.22
query31	2.80	0.41	0.40
query32	3.24	0.50	0.48
query33	2.95	2.96	2.96
query34	16.92	4.32	4.38
query35	4.45	4.43	4.45
query36	0.67	0.53	0.49
query37	0.21	0.18	0.18
query38	0.17	0.16	0.16
query39	0.06	0.05	0.05
query40	0.16	0.14	0.14
query41	0.11	0.06	0.07
query42	0.08	0.07	0.07
query43	0.07	0.07	0.06
Total cold run time: 109.63 s
Total hot run time: 31.61 s

@zddr
Copy link
Contributor Author

zddr commented Aug 20, 2024

run p0

@zddr
Copy link
Contributor Author

zddr commented Aug 20, 2024

run buildall

@zddr
Copy link
Contributor Author

zddr commented Aug 21, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Aug 21, 2024

TPC-H: Total hot run time: 38469 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 96fc5d0f7519d658a068c6c878d164a30d3546d2, data reload: false

------ Round 1 ----------------------------------
q1	18212	4455	4439	4439
q2	2482	208	209	208
q3	12042	1219	1075	1075
q4	10969	795	788	788
q5	8086	2886	2864	2864
q6	265	160	162	160
q7	1000	656	656	656
q8	9367	2086	2113	2086
q9	7135	6565	6577	6565
q10	7038	2184	2278	2184
q11	492	273	282	273
q12	435	263	262	262
q13	18957	2986	3037	2986
q14	300	270	263	263
q15	571	529	524	524
q16	527	403	405	403
q17	1009	653	732	653
q18	7356	6823	6835	6823
q19	6787	1104	1082	1082
q20	750	348	344	344
q21	3823	2809	2929	2809
q22	1131	1040	1022	1022
Total cold run time: 118734 ms
Total hot run time: 38469 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4609	4333	4305	4305
q2	422	300	299	299
q3	2870	2605	2614	2605
q4	1925	1635	1669	1635
q5	5352	5423	5354	5354
q6	233	144	146	144
q7	2060	1705	1687	1687
q8	3185	3350	3340	3340
q9	8420	8396	8398	8396
q10	3425	3161	3201	3161
q11	616	550	521	521
q12	798	624	651	624
q13	17551	3061	3043	3043
q14	320	300	294	294
q15	548	514	522	514
q16	481	460	437	437
q17	1791	1507	1497	1497
q18	7635	7512	7512	7512
q19	1701	1528	1560	1528
q20	2036	1825	1833	1825
q21	9144	5186	4986	4986
q22	1117	1093	1047	1047
Total cold run time: 76239 ms
Total hot run time: 54754 ms

@doris-robot
Copy link

doris-robot commented Aug 21, 2024

TPC-DS: Total hot run time: 191359 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 96fc5d0f7519d658a068c6c878d164a30d3546d2, data reload: false

query1	966	408	396	396
query2	6741	2027	1890	1890
query3	6677	231	238	231
query4	33813	23233	23261	23233
query5	4411	729	700	700
query6	314	222	217	217
query7	4610	328	322	322
query8	471	427	441	427
query9	8679	2559	2522	2522
query10	519	355	340	340
query11	17613	15088	15240	15088
query12	191	139	135	135
query13	1703	469	451	451
query14	10072	7101	7288	7101
query15	280	195	186	186
query16	7686	516	524	516
query17	1693	613	608	608
query18	1952	339	370	339
query19	353	173	170	170
query20	144	138	140	138
query21	248	144	143	143
query22	4300	4185	3953	3953
query23	34273	33448	33254	33254
query24	11244	2907	2872	2872
query25	637	431	423	423
query26	1160	180	181	180
query27	2662	303	305	303
query28	7562	2151	2111	2111
query29	833	470	440	440
query30	341	187	184	184
query31	1073	852	847	847
query32	116	80	82	80
query33	805	349	354	349
query34	906	506	500	500
query35	916	772	761	761
query36	1144	986	959	959
query37	168	106	105	105
query38	3972	3907	3877	3877
query39	1542	1475	1466	1466
query40	240	155	155	155
query41	142	138	139	138
query42	137	118	118	118
query43	542	521	499	499
query44	1264	787	798	787
query45	232	201	196	196
query46	1143	760	773	760
query47	1934	1861	1887	1861
query48	405	344	336	336
query49	1227	589	587	587
query50	864	464	457	457
query51	7256	7016	7154	7016
query52	126	112	113	112
query53	309	226	224	224
query54	925	519	504	504
query55	94	92	93	92
query56	349	325	313	313
query57	1201	1159	1149	1149
query58	332	304	320	304
query59	3023	2959	2766	2766
query60	357	352	338	338
query61	155	152	153	152
query62	861	702	700	700
query63	264	230	232	230
query64	5532	2411	1825	1825
query65	3306	3196	3164	3164
query66	1168	704	668	668
query67	15357	15028	15152	15028
query68	5014	580	604	580
query69	569	316	324	316
query70	1228	1130	1204	1130
query71	505	318	317	317
query72	7376	2380	2083	2083
query73	791	365	359	359
query74	9235	8860	8902	8860
query75	4287	2763	2757	2757
query76	3334	1117	1018	1018
query77	859	444	440	440
query78	9800	9079	9396	9079
query79	3530	557	551	551
query80	1353	614	602	602
query81	632	260	262	260
query82	836	159	164	159
query83	458	226	221	221
query84	297	104	104	104
query85	1581	414	354	354
query86	375	321	331	321
query87	4432	4234	4247	4234
query88	4788	2537	2536	2536
query89	479	330	325	325
query90	2077	231	229	229
query91	154	129	127	127
query92	88	74	79	74
query93	4970	559	545	545
query94	914	338	298	298
query95	393	296	292	292
query96	628	288	281	281
query97	3254	3130	3089	3089
query98	262	233	236	233
query99	1734	1297	1289	1289
Total cold run time: 307768 ms
Total hot run time: 191359 ms

@doris-robot
Copy link

doris-robot commented Aug 21, 2024

ClickBench: Total hot run time: 31.27 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 96fc5d0f7519d658a068c6c878d164a30d3546d2, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.08	0.07
query5	0.52	0.49	0.49
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.04	0.05
query9	0.55	0.50	0.50
query10	0.55	0.55	0.56
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.62	0.60	0.60
query14	0.76	0.82	0.78
query15	0.84	0.82	0.83
query16	0.38	0.36	0.37
query17	0.97	1.04	1.04
query18	0.21	0.20	0.21
query19	1.84	1.80	1.84
query20	0.01	0.01	0.01
query21	15.40	0.67	0.66
query22	3.95	7.04	1.84
query23	18.25	1.51	1.34
query24	2.12	0.22	0.22
query25	0.16	0.09	0.08
query26	0.28	0.19	0.18
query27	0.09	0.09	0.09
query28	13.33	1.05	1.01
query29	12.60	3.39	3.37
query30	0.38	0.21	0.19
query31	2.79	0.39	0.40
query32	3.26	0.48	0.49
query33	2.96	2.98	2.96
query34	17.13	4.32	4.40
query35	4.50	4.50	4.59
query36	0.66	0.50	0.50
query37	0.22	0.18	0.17
query38	0.18	0.17	0.17
query39	0.07	0.06	0.06
query40	0.17	0.15	0.15
query41	0.11	0.07	0.07
query42	0.08	0.07	0.07
query43	0.07	0.06	0.07
Total cold run time: 109.58 s
Total hot run time: 31.27 s

@zddr
Copy link
Contributor Author

zddr commented Aug 21, 2024

run cloud_p1

@morrySnow morrySnow changed the title [fix](mtmv)transfer col in mysql varchar(100) to text when create MTMV [fix](mtmv)transfer col in mysql varchar to text when create MTMV Aug 21, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 21, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 21, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 21, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 8338e18 into apache:master Aug 21, 2024
zddr added a commit to zddr/incubator-doris that referenced this pull request Aug 21, 2024
@zddr
[fix](mtmv) transfer col in mysql varchar to text when create MTMV (a…
10f6016 
…pache#37668)

varchar(100) in mysql represents the ability to store 100 Chinese
characters. but in doris, may need varchar(400)
@zddr zddr mentioned this pull request Aug 21, 2024
Merged
morrySnow pushed a commit that referenced this pull request Aug 22, 2024
@zddr
[fix](mtmv) transfer col in mysql varchar to text when create MTMV (#…
f553645 
…37668) (#39727)

pick from master #37668
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Closed
3 tasks
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@zddr @dataroaring
[fix](mtmv) transfer col in mysql varchar to text when create MTMV (#…
20855b3 
…37668)

varchar(100) in mysql represents the ability to store 100 Chinese
characters. but in doris, may need varchar(400)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

+1 more reviewer

@seawinde seawinde seawinde approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zddr @doris-robot @morrySnow @seawinde