Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 3, 2025

Cherry-picked from #46280

…#46280)

### What problem does this PR solve?

We specified the `excluded_trigger_tables = 'a'` attribute when creating
the materialized view. If table `a` is updated frequently, many invalid
tasks will be generated, and these tasks do not really refresh the mv,
which is unreasonable, too many invalid tasks will wash away useful task
information


Co-authored-by: garenshi <garenshi@tencent.com>
@Thearas
Copy link
Contributor

Thearas commented Jan 3, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Jan 3, 2025
@dataroaring dataroaring reopened this Jan 3, 2025
@Thearas
Copy link
Contributor

Thearas commented Jan 3, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41115 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 845092b0b10d5050ed24195ef03716601fb3fc7b, data reload: false

------ Round 1 ----------------------------------
q1	17619	7799	7320	7320
q2	2777	180	162	162
q3	11522	1156	1208	1156
q4	10390	754	704	704
q5	7845	2915	2870	2870
q6	237	172	144	144
q7	994	614	627	614
q8	9661	2057	2060	2057
q9	6800	6530	6416	6416
q10	7020	2273	2301	2273
q11	461	269	273	269
q12	407	211	212	211
q13	17797	2972	2995	2972
q14	248	217	214	214
q15	568	526	521	521
q16	707	606	617	606
q17	1010	574	664	574
q18	7327	6711	6713	6711
q19	1407	1095	959	959
q20	474	205	194	194
q21	4316	3398	3172	3172
q22	1109	1003	996	996
Total cold run time: 110696 ms
Total hot run time: 41115 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7296	7248	7362	7248
q2	333	252	246	246
q3	3174	3007	2912	2912
q4	2037	1831	1837	1831
q5	5650	5758	5687	5687
q6	217	137	139	137
q7	2254	1842	1793	1793
q8	3319	3504	3486	3486
q9	8961	8933	8964	8933
q10	3650	3534	3509	3509
q11	607	491	496	491
q12	817	614	599	599
q13	5360	3163	3057	3057
q14	279	269	253	253
q15	553	507	500	500
q16	704	670	646	646
q17	1780	1585	1560	1560
q18	7810	7272	7526	7272
q19	1685	1580	1544	1544
q20	2068	1800	1790	1790
q21	5334	5286	5143	5143
q22	1117	1004	1000	1000
Total cold run time: 65005 ms
Total hot run time: 59637 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191867 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 845092b0b10d5050ed24195ef03716601fb3fc7b, data reload: false

query1	979	373	363	363
query2	6521	2052	2093	2052
query3	6706	216	226	216
query4	33805	23564	23444	23444
query5	4389	475	479	475
query6	312	196	183	183
query7	4627	317	312	312
query8	304	230	229	229
query9	9501	2717	2720	2717
query10	478	280	275	275
query11	18044	15197	15111	15111
query12	150	109	108	108
query13	1652	448	420	420
query14	9077	7207	6904	6904
query15	256	174	189	174
query16	8105	448	446	446
query17	1673	573	567	567
query18	2140	312	320	312
query19	365	159	154	154
query20	118	110	108	108
query21	68	47	48	47
query22	4603	4250	4185	4185
query23	34730	33995	34289	33995
query24	11202	2857	2852	2852
query25	672	416	419	416
query26	1478	170	173	170
query27	2801	306	308	306
query28	8017	2477	2458	2458
query29	957	463	441	441
query30	334	174	172	172
query31	1001	823	817	817
query32	96	60	61	60
query33	791	292	320	292
query34	916	495	489	489
query35	942	698	749	698
query36	1104	927	968	927
query37	131	75	74	74
query38	4008	3955	3973	3955
query39	1491	1435	1407	1407
query40	214	87	84	84
query41	54	50	52	50
query42	115	101	97	97
query43	543	478	508	478
query44	1241	816	810	810
query45	185	171	172	171
query46	1128	717	739	717
query47	1949	1893	1866	1866
query48	486	386	376	376
query49	1158	396	377	377
query50	819	409	417	409
query51	7319	7117	7118	7117
query52	99	90	92	90
query53	271	184	191	184
query54	1294	476	467	467
query55	77	78	76	76
query56	257	255	243	243
query57	1225	1122	1121	1121
query58	247	206	211	206
query59	3178	2961	2948	2948
query60	287	247	257	247
query61	113	105	109	105
query62	869	681	670	670
query63	224	194	187	187
query64	5427	647	623	623
query65	3306	3214	3210	3210
query66	1455	327	335	327
query67	15885	15676	15629	15629
query68	4178	588	578	578
query69	418	267	262	262
query70	1175	1178	1101	1101
query71	333	256	252	252
query72	6542	3990	4050	3990
query73	743	343	356	343
query74	10167	8977	8886	8886
query75	3381	2658	2658	2658
query76	2783	1064	1077	1064
query77	426	275	276	275
query78	10579	9636	9663	9636
query79	1107	619	592	592
query80	779	444	422	422
query81	548	242	244	242
query82	1245	121	114	114
query83	241	151	145	145
query84	240	80	84	80
query85	1119	298	290	290
query86	328	294	301	294
query87	4449	4373	4239	4239
query88	3414	2425	2382	2382
query89	401	289	313	289
query90	2149	187	188	187
query91	184	151	148	148
query92	60	52	60	52
query93	1058	559	539	539
query94	816	296	298	296
query95	357	260	260	260
query96	611	284	282	282
query97	3348	3278	3260	3260
query98	217	217	212	212
query99	1496	1321	1281	1281
Total cold run time: 298861 ms
Total hot run time: 191867 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 845092b0b10d5050ed24195ef03716601fb3fc7b, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.52	0.51	0.51
query6	1.13	0.73	0.71
query7	0.02	0.01	0.01
query8	0.04	0.03	0.05
query9	0.56	0.50	0.50
query10	0.55	0.54	0.55
query11	0.14	0.11	0.10
query12	0.15	0.11	0.11
query13	0.62	0.60	0.59
query14	2.94	2.90	2.93
query15	0.90	0.83	0.82
query16	0.39	0.40	0.37
query17	1.07	1.05	1.07
query18	0.24	0.21	0.22
query19	1.87	1.92	1.90
query20	0.01	0.01	0.01
query21	15.36	0.58	0.59
query22	2.36	1.90	1.44
query23	17.07	0.90	0.76
query24	3.38	0.70	1.19
query25	0.31	0.11	0.05
query26	0.50	0.13	0.14
query27	0.06	0.04	0.04
query28	10.63	1.10	1.08
query29	12.55	3.23	3.24
query30	0.27	0.06	0.05
query31	2.85	0.38	0.38
query32	3.25	0.46	0.47
query33	2.98	3.02	3.06
query34	16.72	4.47	4.48
query35	4.50	4.57	4.55
query36	0.67	0.50	0.50
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.09	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.06 s
Total hot run time: 32.32 s

@yiguolei yiguolei merged commit a24a3b2 into branch-3.0 Jan 5, 2025
21 checks passed
@github-actions github-actions bot deleted the auto-pick-46280-branch-3.0 branch January 5, 2025 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants