[Pick](materialized-view) pick some mv fix from master #33365

BiteTheDDDDt · 2024-04-08T07:52:19Z

Proposed changes

#30215 #31962 #33198

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

…d value mv column (apache#30215) support predicate apprear both on key and value mv column

…_array()) case (apache#31962) support mv with bitmap_union(bitmap_from_array()) case

…base slot on mv (apache#33198) fix wrong result when salias name same with base slot on mv

doris-robot · 2024-04-08T07:52:24Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

BiteTheDDDDt · 2024-04-08T07:53:31Z

run buildall

doris-robot · 2024-04-08T08:14:10Z

TPC-H: Total hot run time: 50154 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 379637d38ce3adb500a1494a70076a24bfa8aef5, data reload: false

------ Round 1 ----------------------------------
q1	17544	4395	4382	4382
q2	2036	148	143	143
q3	10270	1894	1920	1894
q4	10098	1271	1313	1271
q5	8515	4075	3953	3953
q6	246	127	122	122
q7	1998	1587	1585	1585
q8	9342	2718	2731	2718
q9	10776	11177	10627	10627
q10	8601	3493	3524	3493
q11	430	237	241	237
q12	463	296	300	296
q13	18350	3999	4043	3999
q14	356	317	326	317
q15	512	467	455	455
q16	697	608	595	595
q17	1143	979	962	962
q18	7344	6937	6830	6830
q19	1648	1533	1502	1502
q20	530	301	291	291
q21	4395	4105	4084	4084
q22	506	401	398	398
Total cold run time: 115800 ms
Total hot run time: 50154 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4268	4279	4268
q2	323	220	221	220
q3	4149	4130	4150	4130
q4	2771	2754	2760	2754
q5	7304	7216	7233	7216
q6	232	118	115	115
q7	3281	2861	2880	2861
q8	4404	4447	4520	4447
q9	17070	17048	16975	16975
q10	4248	4236	4287	4236
q11	739	720	690	690
q12	1014	837	868	837
q13	7197	3771	3707	3707
q14	445	418	426	418
q15	493	462	454	454
q16	762	701	702	701
q17	3897	3845	3825	3825
q18	8834	8704	8788	8704
q19	1706	1703	1665	1665
q20	2401	2155	2081	2081
q21	8585	8446	8482	8446
q22	1038	962	912	912
Total cold run time: 85221 ms
Total hot run time: 79662 ms

doris-robot · 2024-04-08T08:26:44Z

TPC-DS: Total hot run time: 200083 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 379637d38ce3adb500a1494a70076a24bfa8aef5, data reload: false

query1	921	389	378	378
query2	6514	2325	2024	2024
query3	6916	197	192	192
query4	20120	17993	17918	17918
query5	19726	6521	6449	6449
query6	271	210	235	210
query7	4150	302	294	294
query8	253	228	223	223
query9	3143	2679	2603	2603
query10	390	294	298	294
query11	11331	10858	10659	10659
query12	126	72	73	72
query13	5585	665	621	621
query14	17623	13227	13536	13227
query15	391	232	245	232
query16	6450	269	255	255
query17	1757	1465	870	870
query18	2322	401	402	401
query19	201	150	143	143
query20	76	73	74	73
query21	199	94	91	91
query22	5168	5121	5005	5005
query23	32410	31920	31687	31687
query24	6934	6529	6613	6529
query25	523	425	423	423
query26	521	165	156	156
query27	1902	288	290	288
query28	6111	2288	2234	2234
query29	2921	2816	2765	2765
query30	241	163	156	156
query31	891	726	722	722
query32	68	60	60	60
query33	388	234	243	234
query34	838	456	468	456
query35	1103	880	898	880
query36	1251	1267	1159	1159
query37	90	61	59	59
query38	3107	2915	2912	2912
query39	1369	1326	1315	1315
query40	205	85	94	85
query41	35	32	35	32
query42	84	82	83	82
query43	644	611	548	548
query44	1092	718	719	718
query45	232	226	220	220
query46	1239	981	965	965
query47	1927	1726	1967	1726
query48	957	676	656	656
query49	605	358	372	358
query50	857	629	619	619
query51	4745	4664	4624	4624
query52	95	78	74	74
query53	429	322	319	319
query54	2645	2444	2438	2438
query55	93	78	83	78
query56	213	193	203	193
query57	1255	1218	1225	1218
query58	208	207	187	187
query59	3483	3176	3327	3176
query60	201	195	200	195
query61	86	86	81	81
query62	857	458	482	458
query63	470	328	327	327
query64	2531	1418	1341	1341
query65	3616	3558	3541	3541
query66	797	366	359	359
query67	17007	15389	14736	14736
query68	10240	654	673	654
query69	574	334	361	334
query70	1899	1398	1386	1386
query71	414	293	307	293
query72	6578	3394	3403	3394
query73	734	316	322	316
query74	6300	5918	5891	5891
query75	5308	3622	3606	3606
query76	6704	1203	1201	1201
query77	1155	255	252	252
query78	12522	11441	12084	11441
query79	6587	644	633	633
query80	886	380	395	380
query81	488	233	230	230
query82	1337	100	98	98
query83	174	132	135	132
query84	253	69	69	69
query85	864	286	286	286
query86	330	318	295	295
query87	3213	3008	3011	3008
query88	4618	2358	2348	2348
query89	363	298	284	284
query90	1934	211	198	198
query91	148	117	116	116
query92	57	49	51	49
query93	5489	567	551	551
query94	707	203	203	203
query95	1099	1062	1052	1052
query96	644	326	344	326
query97	6376	6367	6339	6339
query98	182	168	173	168
query99	2948	927	935	927
Total cold run time: 312034 ms
Total hot run time: 200083 ms

doris-robot · 2024-04-08T08:32:12Z

ClickBench: Total hot run time: 30.93 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 379637d38ce3adb500a1494a70076a24bfa8aef5, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.80	0.07	0.07
query5	0.53	0.52	0.52
query6	1.25	0.60	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.48	0.48
query10	0.53	0.54	0.55
query11	0.13	0.09	0.09
query12	0.12	0.09	0.09
query13	0.64	0.61	0.62
query14	0.79	0.80	0.79
query15	0.79	0.77	0.75
query16	0.36	0.36	0.37
query17	1.03	1.02	1.01
query18	0.23	0.25	0.24
query19	1.92	1.82	1.82
query20	0.01	0.01	0.01
query21	15.48	0.54	0.55
query22	2.48	2.18	1.53
query23	17.62	0.98	0.93
query24	5.08	1.32	1.68
query25	0.42	0.09	0.06
query26	0.66	0.15	0.16
query27	0.05	0.04	0.03
query28	7.21	0.78	0.70
query29	12.68	2.26	2.34
query30	0.62	0.52	0.51
query31	2.81	0.40	0.38
query32	3.37	0.49	0.49
query33	3.09	3.08	3.10
query34	15.57	4.78	4.78
query35	4.85	4.82	4.83
query36	1.08	1.01	1.03
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.13	0.14
query41	0.07	0.02	0.02
query42	0.02	0.02	0.02
query43	0.03	0.02	0.01
Total cold run time: 104.49 s
Total hot run time: 30.93 s

doris-robot · 2024-04-08T08:35:27Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 379637d38ce3adb500a1494a70076a24bfa8aef5 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

BiteTheDDDDt · 2024-04-08T09:05:02Z

run buildall

doris-robot · 2024-04-08T09:33:11Z

TPC-H: Total hot run time: 49924 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41ecb51b2763b91c17b0a68b5a654b90c3185734, data reload: false

------ Round 1 ----------------------------------
q1	17631	4399	4352	4352
q2	2047	148	137	137
q3	10388	1905	1953	1905
q4	10371	1231	1311	1231
q5	8296	3945	3962	3945
q6	233	122	123	122
q7	2047	1594	1595	1594
q8	9473	2689	2704	2689
q9	11230	10512	10488	10488
q10	8651	3487	3521	3487
q11	430	242	246	242
q12	468	293	300	293
q13	18330	3951	4048	3951
q14	365	321	325	321
q15	514	455	462	455
q16	678	596	601	596
q17	1116	942	949	942
q18	7335	6880	6869	6869
q19	1670	1570	1505	1505
q20	509	287	319	287
q21	4471	4109	4134	4109
q22	509	413	404	404
Total cold run time: 116762 ms
Total hot run time: 49924 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4315	4336	4315
q2	320	222	216	216
q3	4134	4140	4138	4138
q4	2739	2736	2733	2733
q5	7280	7160	7187	7160
q6	229	119	115	115
q7	3242	2823	2845	2823
q8	4356	4491	4570	4491
q9	17165	17003	17107	17003
q10	4246	4271	4276	4271
q11	766	683	684	683
q12	1045	837	853	837
q13	7547	3789	3766	3766
q14	459	439	419	419
q15	511	456	462	456
q16	766	698	710	698
q17	3840	3847	3900	3847
q18	9212	9396	9364	9364
q19	1730	1678	1678	1678
q20	2432	2171	2135	2135
q21	8656	8663	8503	8503
q22	1040	940	963	940
Total cold run time: 86049 ms
Total hot run time: 80591 ms

doris-robot · 2024-04-08T09:45:48Z

TPC-DS: Total hot run time: 201572 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41ecb51b2763b91c17b0a68b5a654b90c3185734, data reload: false

query1	916	390	374	374
query2	6539	2212	1982	1982
query3	6926	199	199	199
query4	20040	18022	17977	17977
query5	19728	6526	6472	6472
query6	352	213	236	213
query7	4607	299	305	299
query8	248	238	215	215
query9	3108	2688	2616	2616
query10	431	300	303	300
query11	11405	10667	10742	10667
query12	121	80	67	67
query13	5572	667	641	641
query14	18414	13537	13347	13347
query15	365	226	240	226
query16	6437	268	257	257
query17	1375	1567	874	874
query18	2282	411	403	403
query19	202	148	146	146
query20	74	75	75	75
query21	185	93	91	91
query22	5396	5144	5214	5144
query23	32452	31841	31813	31813
query24	6741	6547	6523	6523
query25	535	433	412	412
query26	514	164	155	155
query27	1779	295	292	292
query28	6139	2278	2231	2231
query29	2868	2754	2791	2754
query30	246	160	162	160
query31	891	743	729	729
query32	72	57	58	57
query33	404	239	242	239
query34	848	456	470	456
query35	1109	895	900	895
query36	1397	1044	1091	1044
query37	92	60	60	60
query38	3092	2933	2884	2884
query39	1377	1329	1331	1329
query40	206	92	99	92
query41	39	32	31	31
query42	93	94	85	85
query43	668	614	608	608
query44	1114	721	722	721
query45	239	228	221	221
query46	1227	942	958	942
query47	1870	1676	1626	1626
query48	975	690	688	688
query49	601	372	374	372
query50	874	629	578	578
query51	4728	4713	4664	4664
query52	78	72	76	72
query53	454	310	312	310
query54	2646	2460	2408	2408
query55	85	83	83	83
query56	215	205	205	205
query57	1148	1206	1100	1100
query58	208	204	191	191
query59	3532	3144	3234	3144
query60	209	202	196	196
query61	84	85	87	85
query62	796	495	486	486
query63	466	337	342	337
query64	2474	1515	1371	1371
query65	3639	3552	3519	3519
query66	750	364	363	363
query67	17541	15519	15816	15519
query68	9080	663	683	663
query69	576	337	345	337
query70	1781	1480	1476	1476
query71	419	312	306	306
query72	6565	3423	3599	3423
query73	735	320	327	320
query74	6334	5858	5869	5858
query75	5370	3742	3671	3671
query76	5747	1153	1216	1153
query77	978	273	243	243
query78	12653	11781	11983	11781
query79	8596	654	642	642
query80	1271	407	393	393
query81	479	236	231	231
query82	1431	95	96	95
query83	169	133	130	130
query84	251	68	67	67
query85	854	293	291	291
query86	344	328	317	317
query87	3215	3046	2984	2984
query88	4709	2352	2369	2352
query89	369	313	274	274
query90	1952	199	199	199
query91	149	116	116	116
query92	56	54	50	50
query93	5639	610	570	570
query94	668	201	195	195
query95	1093	1039	1060	1039
query96	654	326	320	320
query97	6475	6406	6454	6406
query98	188	175	169	169
query99	2935	866	907	866
Total cold run time: 313973 ms
Total hot run time: 201572 ms

doris-robot · 2024-04-08T09:51:15Z

ClickBench: Total hot run time: 31.18 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41ecb51b2763b91c17b0a68b5a654b90c3185734, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.04	0.05
query4	1.81	0.07	0.06
query5	0.52	0.53	0.52
query6	1.35	0.61	0.61
query7	0.01	0.01	0.02
query8	0.03	0.02	0.03
query9	0.53	0.49	0.48
query10	0.55	0.53	0.53
query11	0.13	0.08	0.09
query12	0.11	0.09	0.10
query13	0.62	0.62	0.61
query14	0.78	0.77	0.80
query15	0.77	0.77	0.78
query16	0.36	0.37	0.37
query17	1.01	1.01	1.00
query18	0.21	0.29	0.22
query19	1.93	1.85	1.83
query20	0.02	0.01	0.02
query21	15.47	0.54	0.55
query22	2.01	2.29	1.75
query23	17.23	0.98	1.00
query24	6.74	1.87	1.33
query25	0.41	0.10	0.05
query26	0.78	0.17	0.16
query27	0.05	0.03	0.03
query28	5.27	0.77	0.77
query29	12.70	2.26	2.21
query30	0.60	0.53	0.51
query31	2.82	0.40	0.37
query32	3.38	0.52	0.50
query33	3.06	3.06	3.09
query34	15.26	4.77	4.77
query35	4.85	4.86	4.82
query36	1.06	1.02	1.01
query37	0.06	0.05	0.05
query38	0.03	0.03	0.03
query39	0.01	0.01	0.02
query40	0.16	0.14	0.13
query41	0.06	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.01
Total cold run time: 103.11 s
Total hot run time: 31.18 s

doris-robot · 2024-04-08T09:54:31Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 41ecb51b2763b91c17b0a68b5a654b90c3185734 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.0 seconds inserted 10000000 Rows, about 476K ops/s

…pache#31962 apache#33198 (apache#33365)

BiteTheDDDDt added 3 commits April 8, 2024 15:47

[Feature](materialized-view) support predicate apprear both on key an…

96f923e

…d value mv column (apache#30215) support predicate apprear both on key and value mv column

[Feature](materialized-view) support mv with bitmap_union(bitmap_from…

afda2cf

…_array()) case (apache#31962) support mv with bitmap_union(bitmap_from_array()) case

[Bug](materialized-view) fix wrong result when salias name same with …

379637d

…base slot on mv (apache#33198) fix wrong result when salias name same with base slot on mv

github-actions bot added area/nereids area/planner Issues or PRs related to the query planner kind/test labels Apr 8, 2024

remove

41ecb51

xiaokang merged commit d4d213f into apache:branch-2.0 Apr 17, 2024

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[Pick](materialized-view) pick some mv fix from master apache#30215 a…

d850892

…pache#31962 apache#33198 (apache#33365)

BiteTheDDDDt deleted the cp_0408_3 branch January 20, 2025 06:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Pick](materialized-view) pick some mv fix from master #33365

[Pick](materialized-view) pick some mv fix from master #33365

Uh oh!

BiteTheDDDDt commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

BiteTheDDDDt commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

BiteTheDDDDt commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[Pick](materialized-view) pick some mv fix from master #33365

[Pick](materialized-view) pick some mv fix from master #33365

Uh oh!

Conversation

BiteTheDDDDt commented Apr 8, 2024

Proposed changes

Further comments

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

BiteTheDDDDt commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

BiteTheDDDDt commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

doris-robot commented Apr 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants