Skip to content

Pick some pr from master to 3.0 (#40871) (#41145) #42303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 2 commits into from
Oct 23, 2024
Merged

Pick some pr from master to 3.0 (#40871) (#41145) #42303

dataroaring merged 2 commits into from
Oct 23, 2024

Conversation

@seawinde
Copy link
Contributor

@seawinde seawinde commented Oct 23, 2024

Proposed changes

pr: #40871
commitId: da6ac0c

pr: #41145
commitId: 5e6e4bf

@seawinde
[test](mtmv) Fix regression test unstable (apache#40871)
d31485a 
1. Optimized `waitForRollUpJob` method in `Suite.groovy` to make sure
roll up is build succesfully before rewirte.
2. Modify expect rewritten result in` agg_sync_mv.groovy`
@seawinde
[fix](mtmv) regression test unstable and error (apache#41145)
b07fbb2 
This is brought by apache#39265

1. Some test fail because `MaterializedViewRewriteSuccessButNotChose`,
because in these tests materialized view are not analyzed after create,
So Add `analyze clause` when waite materialized view task finished
2. No signature of method:
org.apache.doris.regression.suite.Suite.check_mv_rewrite_success(),
because `check_mv_rewrite_success` is not available in Suite.groovy. Fix
this by using correct method.
3. If partition row count statistics is not ready, should not check mv
rewrte by CBO succesfully
@doris-robot
Copy link

doris-robot commented Oct 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@seawinde
Copy link
Contributor Author

seawinde commented Oct 23, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 23, 2024

TPC-H: Total hot run time: 40916 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b07fbb20694e0c1fb1850317bb9a7cf1fb9b0579, data reload: false

------ Round 1 ----------------------------------
q1	17578	7483	7367	7367
q2	2020	274	268	268
q3	12138	1068	1169	1068
q4	10572	753	747	747
q5	7756	2860	2855	2855
q6	238	150	149	149
q7	1002	614	621	614
q8	9356	1945	1970	1945
q9	6617	6461	6377	6377
q10	6971	2289	2340	2289
q11	442	243	250	243
q12	414	228	224	224
q13	17810	2970	2969	2969
q14	237	215	212	212
q15	575	525	579	525
q16	686	624	610	610
q17	991	558	561	558
q18	7302	6571	6595	6571
q19	1945	1063	1026	1026
q20	581	272	273	272
q21	4039	3277	3042	3042
q22	1095	985	993	985
Total cold run time: 110365 ms
Total hot run time: 40916 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7298	7247	7226	7226
q2	314	224	229	224
q3	3067	2879	2914	2879
q4	2058	1822	1842	1822
q5	5764	5715	5726	5715
q6	241	148	149	148
q7	2188	1809	1825	1809
q8	3350	3537	3469	3469
q9	8998	8976	9104	8976
q10	3567	3609	3524	3524
q11	584	471	475	471
q12	812	624	624	624
q13	16578	3122	3132	3122
q14	296	270	281	270
q15	589	530	513	513
q16	721	709	685	685
q17	1870	1628	1600	1600
q18	8378	7760	7524	7524
q19	6072	1555	1614	1555
q20	2166	1864	1868	1864
q21	5434	5205	5288	5205
q22	1110	1027	1026	1026
Total cold run time: 81455 ms
Total hot run time: 60251 ms

@doris-robot
Copy link

doris-robot commented Oct 23, 2024

TPC-DS: Total hot run time: 189750 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b07fbb20694e0c1fb1850317bb9a7cf1fb9b0579, data reload: false

query1	969	393	380	380
query2	6235	2138	2001	2001
query3	8687	210	198	198
query4	33703	23405	23360	23360
query5	3766	479	474	474
query6	278	169	162	162
query7	4200	317	302	302
query8	256	216	210	210
query9	9566	2682	2632	2632
query10	465	285	283	283
query11	17814	15103	15099	15099
query12	143	100	98	98
query13	1540	417	406	406
query14	8900	7246	7324	7246
query15	218	177	172	172
query16	7717	493	469	469
query17	1612	596	599	596
query18	2002	330	303	303
query19	213	163	157	157
query20	124	124	112	112
query21	215	110	110	110
query22	4603	4434	4329	4329
query23	35364	33629	34127	33629
query24	11347	2841	2859	2841
query25	596	416	411	411
query26	724	162	170	162
query27	2103	295	292	292
query28	6596	2502	2478	2478
query29	796	439	440	439
query30	284	152	159	152
query31	1035	780	800	780
query32	98	57	56	56
query33	742	286	294	286
query34	917	499	495	495
query35	828	727	741	727
query36	1104	930	943	930
query37	145	86	85	85
query38	3985	3874	3805	3805
query39	1476	1456	1419	1419
query40	204	100	98	98
query41	51	50	47	47
query42	124	96	98	96
query43	535	495	486	486
query44	1220	780	793	780
query45	199	164	165	164
query46	1158	744	722	722
query47	1882	1778	1801	1778
query48	468	368	371	368
query49	919	396	390	390
query50	820	410	413	410
query51	7002	7012	6854	6854
query52	113	90	91	90
query53	258	190	183	183
query54	1209	495	470	470
query55	81	77	78	77
query56	286	259	265	259
query57	1181	1126	1115	1115
query58	242	238	238	238
query59	3106	3030	2976	2976
query60	291	279	258	258
query61	102	106	139	106
query62	861	647	678	647
query63	215	186	185	185
query64	3749	659	616	616
query65	3274	3147	3153	3147
query66	810	314	311	311
query67	15785	15380	15232	15232
query68	4552	540	539	539
query69	616	279	301	279
query70	1130	1120	1131	1120
query71	449	288	268	268
query72	7984	3896	3892	3892
query73	757	337	345	337
query74	10319	8932	8885	8885
query75	4490	2661	2630	2630
query76	3756	949	795	795
query77	790	285	278	278
query78	9916	9293	9107	9107
query79	4047	594	579	579
query80	1403	447	439	439
query81	570	243	247	243
query82	936	142	142	142
query83	396	146	136	136
query84	299	85	80	80
query85	1984	302	288	288
query86	458	294	300	294
query87	4457	4259	4336	4259
query88	4164	2430	2439	2430
query89	522	298	282	282
query90	2141	184	180	180
query91	182	144	142	142
query92	65	50	46	46
query93	5749	545	537	537
query94	1129	294	274	274
query95	347	249	249	249
query96	643	278	279	278
query97	3352	3086	3157	3086
query98	213	199	187	187
query99	1558	1312	1280	1280
Total cold run time: 306885 ms
Total hot run time: 189750 ms

@seawinde seawinde changed the title Pick some pr from master 30 #40871 #41145 Pick some pr from master to 3.0 #40871 #41145 Oct 23, 2024
@seawinde seawinde changed the title Pick some pr from master to 3.0 #40871 #41145 Pick some pr from master to 3.0 (#40871) #41145 Oct 23, 2024
@seawinde seawinde changed the title Pick some pr from master to 3.0 (#40871) #41145 Pick some pr from master to 3.0 (#40871) (#41145) Oct 23, 2024
@dataroaring dataroaring merged commit fd4e9a8 into apache:branch-3.0 Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@seawinde @doris-robot @dataroaring