Skip to content

branch-3.0: [fix](mtmv) Fix data wrong when query with table operator such as TABLESAMPLE or tablet and so on #43030 #44873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Dec 6, 2024
Merged

branch-3.0: [fix](mtmv) Fix data wrong when query with table operator such as TABLESAMPLE or tablet and so on #43030 #44873

morrySnow merged 1 commit into from
Dec 6, 2024

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Dec 2, 2024

Cherry-picked from #43030

@seawinde
[fix](mtmv) Fix data wrong when query with table operator such as TAB…
e0fe887 
…LESAMPLE or tablet and so on (#43030)

Related PR: #28064

Materialized view is as following:

        CREATE MATERIALIZED VIEW mv1
        BUILD IMMEDIATE REFRESH AUTO ON MANUAL  
        DISTRIBUTED BY RANDOM BUCKETS 2  
        PROPERTIES ('replication_num' = '1')  
        AS 
       select * from orders

If run query as following, should rewrite fail by materialized view
above to make sure data correctness

select * from orders TABLET(110);
select * from orders index query_index_test;
select * from orders TABLESAMPLE(20 percent);
select * from orders_partition PARTITION (day_2);

At before, this would rewrite by materialized view succesfully and the
result data is wrong, This pr fix this.
@doris-robot
Copy link

doris-robot commented Dec 2, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Dec 2, 2024
@dataroaring dataroaring reopened this Dec 2, 2024
@doris-robot
Copy link

doris-robot commented Dec 2, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Dec 2, 2024

TPC-H: Total hot run time: 40352 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e0fe887c5b5c7fb93e79ad1d45189eb1d218aaa6, data reload: false

------ Round 1 ----------------------------------
q1	17581	7320	7239	7239
q2	2056	171	162	162
q3	10672	1072	1128	1072
q4	10562	785	660	660
q5	7724	2853	2817	2817
q6	230	145	144	144
q7	945	611	603	603
q8	9562	1896	1973	1896
q9	7250	6377	6418	6377
q10	7000	2253	2329	2253
q11	452	266	255	255
q12	403	209	204	204
q13	17798	3107	3109	3107
q14	243	213	215	213
q15	575	520	527	520
q16	653	619	620	619
q17	985	605	500	500
q18	7248	6553	6534	6534
q19	1668	973	993	973
q20	489	199	209	199
q21	3998	3041	3084	3041
q22	1049	993	964	964
Total cold run time: 109143 ms
Total hot run time: 40352 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7279	7204	7190	7190
q2	323	230	232	230
q3	2859	2843	2839	2839
q4	2023	1824	1728	1728
q5	5653	5697	5657	5657
q6	223	136	137	136
q7	2190	1766	1796	1766
q8	3304	3497	3487	3487
q9	8733	8900	8842	8842
q10	3513	3519	3489	3489
q11	602	507	488	488
q12	805	630	604	604
q13	16477	3136	3140	3136
q14	310	271	264	264
q15	569	518	525	518
q16	714	668	670	668
q17	1855	1613	1572	1572
q18	8186	7941	7412	7412
q19	5507	1533	1611	1533
q20	2066	1823	1832	1823
q21	5427	5197	5387	5197
q22	1061	1018	989	989
Total cold run time: 79679 ms
Total hot run time: 59568 ms

@doris-robot
Copy link

doris-robot commented Dec 2, 2024

TPC-DS: Total hot run time: 197085 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e0fe887c5b5c7fb93e79ad1d45189eb1d218aaa6, data reload: false

query1	1252	924	919	919
query2	6246	2114	2031	2031
query3	10836	4115	4116	4115
query4	68705	28880	23492	23492
query5	5016	438	452	438
query6	413	173	193	173
query7	5664	312	336	312
query8	310	223	228	223
query9	9334	2669	2658	2658
query10	490	291	273	273
query11	17531	15320	15603	15320
query12	166	107	104	104
query13	1550	435	437	435
query14	10646	7428	7522	7428
query15	215	176	172	172
query16	7346	487	483	483
query17	1147	598	592	592
query18	1841	329	337	329
query19	246	144	145	144
query20	121	114	109	109
query21	217	99	107	99
query22	4746	4556	4473	4473
query23	34560	34071	34111	34071
query24	6130	2876	2927	2876
query25	503	396	393	393
query26	675	163	165	163
query27	1853	305	300	300
query28	4124	2574	2545	2545
query29	665	448	446	446
query30	240	161	162	161
query31	992	819	825	819
query32	68	56	51	51
query33	460	308	270	270
query34	905	510	505	505
query35	859	755	723	723
query36	1101	966	961	961
query37	124	69	77	69
query38	4094	4046	3968	3968
query39	1508	1491	1496	1491
query40	209	103	96	96
query41	54	51	53	51
query42	108	102	99	99
query43	539	510	513	510
query44	1135	793	817	793
query45	187	174	169	169
query46	1177	751	735	735
query47	1999	1859	1910	1859
query48	459	382	374	374
query49	747	392	400	392
query50	828	417	425	417
query51	7435	7294	7149	7149
query52	105	92	96	92
query53	255	178	182	178
query54	579	465	457	457
query55	76	77	74	74
query56	258	244	251	244
query57	1203	1103	1099	1099
query58	219	209	210	209
query59	3168	3177	3101	3101
query60	287	259	260	259
query61	129	129	127	127
query62	781	647	648	647
query63	206	189	188	188
query64	1540	672	670	670
query65	3254	3159	3174	3159
query66	706	297	296	296
query67	15793	15442	15418	15418
query68	5053	589	569	569
query69	429	248	255	248
query70	1142	1135	1106	1106
query71	403	250	247	247
query72	6451	4053	3873	3873
query73	766	343	344	343
query74	10204	8997	8908	8908
query75	3342	2652	2632	2632
query76	2124	1082	1042	1042
query77	552	281	273	273
query78	10682	9787	9469	9469
query79	8326	587	587	587
query80	1918	436	430	430
query81	564	242	240	240
query82	1029	111	120	111
query83	277	139	140	139
query84	290	80	79	79
query85	1653	297	328	297
query86	447	300	299	299
query87	4451	4235	4221	4221
query88	5460	2379	2424	2379
query89	540	295	289	289
query90	2194	182	183	182
query91	184	143	143	143
query92	61	47	46	46
query93	6555	558	550	550
query94	918	280	286	280
query95	340	255	250	250
query96	629	281	275	275
query97	3388	3117	3161	3117
query98	212	200	204	200
query99	1634	1276	1265	1265
Total cold run time: 338553 ms
Total hot run time: 197085 ms

@morrySnow morrySnow merged commit 4952a7c into branch-3.0 Dec 6, 2024
12 checks passed
@github-actions github-actions bot deleted the auto-pick-43030-branch-3.0 branch December 6, 2024 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@doris-robot @dataroaring @morrySnow @seawinde