Skip to content

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Oct 16, 2024

Followup #38244
When setting use_meta_cache=true, the include_database_list and exclude_database_list
will have no effect. This PR fix it.

Also fix a bug, related to #40479.
if include_database_list and exclude_database_list is set, and use a database which is excluded,
Doris should return "database not found" error

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zy-kkk
Copy link
Member

zy-kkk commented Oct 16, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41126 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 260975e27ac69ebb2e1c84ba61e28e09a0df399b, data reload: false

------ Round 1 ----------------------------------
q1	17589	7368	7198	7198
q2	2030	272	283	272
q3	12035	1042	1142	1042
q4	10576	836	860	836
q5	7766	3067	3063	3063
q6	238	151	147	147
q7	1009	625	609	609
q8	9348	1956	1947	1947
q9	6963	6444	6369	6369
q10	7036	2439	2436	2436
q11	442	251	247	247
q12	409	218	216	216
q13	17794	2986	3003	2986
q14	251	222	211	211
q15	573	513	528	513
q16	651	588	574	574
q17	963	591	446	446
q18	7220	6699	6775	6699
q19	1351	956	973	956
q20	465	182	175	175
q21	3916	3193	3277	3193
q22	1124	1010	991	991
Total cold run time: 109749 ms
Total hot run time: 41126 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7221	7202	7213	7202
q2	325	225	224	224
q3	3026	2981	2929	2929
q4	2131	1832	1778	1778
q5	5725	5745	5763	5745
q6	248	147	147	147
q7	2270	1880	1798	1798
q8	3352	3524	3400	3400
q9	8896	8918	8913	8913
q10	3579	3600	3574	3574
q11	591	489	488	488
q12	831	647	657	647
q13	10069	3220	3227	3220
q14	309	278	269	269
q15	574	523	530	523
q16	693	643	638	638
q17	1830	1609	1596	1596
q18	8211	7801	7640	7640
q19	1735	1382	1602	1382
q20	2104	1855	1863	1855
q21	5572	5383	5341	5341
q22	1181	1046	1071	1046
Total cold run time: 70473 ms
Total hot run time: 60355 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192242 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 260975e27ac69ebb2e1c84ba61e28e09a0df399b, data reload: false

query1	869	391	404	391
query2	6237	2095	2012	2012
query3	8695	201	202	201
query4	33890	23539	23699	23539
query5	3508	487	489	487
query6	264	169	162	162
query7	4188	295	297	295
query8	280	228	228	228
query9	9393	2758	2741	2741
query10	454	284	265	265
query11	17789	15323	15551	15323
query12	149	100	105	100
query13	1555	439	404	404
query14	9921	7721	7600	7600
query15	269	172	184	172
query16	7812	474	471	471
query17	1606	600	601	600
query18	2107	312	312	312
query19	391	164	162	162
query20	122	115	114	114
query21	211	107	111	107
query22	4762	4656	4496	4496
query23	35131	34149	34206	34149
query24	10902	2742	2725	2725
query25	610	400	395	395
query26	1056	161	162	161
query27	2244	280	289	280
query28	7520	2428	2416	2416
query29	785	425	416	416
query30	260	153	160	153
query31	1041	761	802	761
query32	99	56	56	56
query33	772	299	289	289
query34	909	515	533	515
query35	902	764	759	759
query36	1099	940	968	940
query37	159	91	92	91
query38	4095	3899	4045	3899
query39	1507	1418	1422	1418
query40	216	100	100	100
query41	48	47	46	46
query42	120	102	102	102
query43	528	477	490	477
query44	1340	821	814	814
query45	200	166	167	166
query46	1147	697	708	697
query47	1950	1835	1861	1835
query48	416	325	324	324
query49	918	433	432	432
query50	812	389	406	389
query51	7004	6991	7016	6991
query52	104	94	91	91
query53	267	186	183	183
query54	1284	430	438	430
query55	80	80	80	80
query56	292	274	278	274
query57	1316	1160	1149	1149
query58	235	229	255	229
query59	3216	3024	2968	2968
query60	284	262	268	262
query61	106	103	104	103
query62	845	681	662	662
query63	245	198	184	184
query64	3995	630	599	599
query65	3291	3225	3203	3203
query66	879	295	296	295
query67	16241	15537	15736	15537
query68	4307	587	542	542
query69	532	290	290	290
query70	1201	1126	1107	1107
query71	363	272	299	272
query72	7341	3941	3977	3941
query73	782	356	356	356
query74	10193	9132	8971	8971
query75	3467	2658	2688	2658
query76	2980	930	845	845
query77	615	301	283	283
query78	10499	9642	9519	9519
query79	2737	615	614	614
query80	2761	441	451	441
query81	576	240	233	233
query82	714	143	141	141
query83	303	134	131	131
query84	286	68	71	68
query85	1452	305	290	290
query86	425	283	297	283
query87	4467	4336	4263	4263
query88	3666	2211	2168	2168
query89	411	284	292	284
query90	1968	184	182	182
query91	136	99	100	99
query92	58	48	48	48
query93	1522	534	535	534
query94	1037	293	266	266
query95	352	249	246	246
query96	607	282	281	281
query97	3302	3152	3159	3152
query98	225	203	197	197
query99	1562	1310	1315	1310
Total cold run time: 301789 ms
Total hot run time: 192242 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.09 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 260975e27ac69ebb2e1c84ba61e28e09a0df399b, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.03	0.03
query3	0.23	0.06	0.07
query4	1.65	0.10	0.10
query5	0.51	0.51	0.50
query6	1.13	0.72	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.49	0.50
query10	0.54	0.54	0.54
query11	0.14	0.11	0.12
query12	0.14	0.11	0.11
query13	0.59	0.59	0.60
query14	2.76	2.74	2.86
query15	0.90	0.83	0.83
query16	0.40	0.37	0.39
query17	1.05	1.05	1.08
query18	0.20	0.20	0.20
query19	1.89	1.79	2.04
query20	0.02	0.01	0.01
query21	15.37	0.58	0.58
query22	2.99	2.60	1.70
query23	16.88	0.89	0.93
query24	3.37	0.48	2.49
query25	0.38	0.12	0.10
query26	0.46	0.13	0.14
query27	0.03	0.05	0.03
query28	9.76	1.11	1.07
query29	12.57	3.18	3.22
query30	0.24	0.06	0.06
query31	2.88	0.38	0.38
query32	3.29	0.46	0.45
query33	2.97	3.03	3.01
query34	16.78	4.43	4.47
query35	4.50	4.51	4.52
query36	0.65	0.48	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.13
query41	0.07	0.02	0.03
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.43 s
Total hot run time: 32.09 s

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41296 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc5a183809529fd2eb07b31731896e2bc13ecfaa, data reload: false

------ Round 1 ----------------------------------
q1	17587	7950	7254	7254
q2	2033	279	274	274
q3	12103	1069	1117	1069
q4	10591	831	884	831
q5	7758	3061	3057	3057
q6	247	147	146	146
q7	1040	617	594	594
q8	9354	1925	1973	1925
q9	6568	6442	6388	6388
q10	7054	2404	2410	2404
q11	458	255	249	249
q12	404	222	220	220
q13	17803	3025	2993	2993
q14	237	217	213	213
q15	564	520	518	518
q16	657	576	580	576
q17	956	520	494	494
q18	7241	6801	6839	6801
q19	1355	949	951	949
q20	500	184	182	182
q21	3973	3252	3154	3154
q22	1137	1005	1018	1005
Total cold run time: 109620 ms
Total hot run time: 41296 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7228	7237	7203	7203
q2	324	225	226	225
q3	3042	3008	2957	2957
q4	2043	1763	1822	1763
q5	5738	5793	5825	5793
q6	225	147	143	143
q7	2233	1823	1787	1787
q8	3384	3500	3477	3477
q9	8879	8912	8894	8894
q10	3632	3595	3540	3540
q11	589	480	491	480
q12	855	656	597	597
q13	8907	3164	3177	3164
q14	304	288	277	277
q15	578	525	515	515
q16	684	635	631	631
q17	1843	1624	1590	1590
q18	8168	7755	7580	7580
q19	1700	1471	1458	1458
q20	2119	1886	1850	1850
q21	5353	5386	5299	5299
q22	1128	1083	1074	1074
Total cold run time: 68956 ms
Total hot run time: 60297 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190392 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc5a183809529fd2eb07b31731896e2bc13ecfaa, data reload: false

query1	925	397	392	392
query2	6277	2049	2019	2019
query3	8686	191	196	191
query4	34226	23782	23558	23558
query5	3481	467	450	450
query6	259	168	155	155
query7	4200	294	295	294
query8	288	224	218	218
query9	9459	2728	2707	2707
query10	480	265	267	265
query11	17885	15251	15166	15166
query12	153	101	96	96
query13	1557	430	402	402
query14	8688	6570	7400	6570
query15	267	171	182	171
query16	7910	424	451	424
query17	1620	553	554	553
query18	1962	327	316	316
query19	376	160	160	160
query20	134	133	119	119
query21	214	112	118	112
query22	4848	4611	4560	4560
query23	37045	33962	34077	33962
query24	11340	2725	2688	2688
query25	531	384	379	379
query26	1009	152	153	152
query27	2667	286	287	286
query28	7834	2420	2399	2399
query29	646	411	414	411
query30	260	152	143	143
query31	1052	793	781	781
query32	92	50	55	50
query33	741	289	273	273
query34	928	514	499	499
query35	897	734	764	734
query36	1145	960	948	948
query37	147	93	91	91
query38	4145	3844	3901	3844
query39	1490	1408	1460	1408
query40	256	95	98	95
query41	45	42	41	41
query42	122	94	94	94
query43	520	481	482	481
query44	1250	820	808	808
query45	188	157	165	157
query46	1132	681	677	677
query47	1988	1845	1815	1815
query48	428	316	307	307
query49	979	409	409	409
query50	810	384	393	384
query51	7124	6896	6910	6896
query52	98	89	88	88
query53	249	175	176	175
query54	1211	422	423	422
query55	80	72	78	72
query56	281	263	251	251
query57	1287	1143	1143	1143
query58	235	232	241	232
query59	3178	2982	3040	2982
query60	294	260	269	260
query61	102	98	100	98
query62	864	687	676	676
query63	221	185	178	178
query64	4864	709	687	687
query65	3308	3216	3201	3201
query66	1345	313	316	313
query67	15938	15659	15551	15551
query68	3529	589	610	589
query69	420	288	280	280
query70	1180	1074	1100	1074
query71	336	275	281	275
query72	5666	3994	3792	3792
query73	770	363	372	363
query74	9826	8951	9087	8951
query75	3378	2655	2638	2638
query76	2033	871	869	869
query77	406	283	290	283
query78	10383	9592	9563	9563
query79	1142	594	596	594
query80	1474	443	456	443
query81	561	236	234	234
query82	257	144	143	143
query83	245	136	130	130
query84	258	69	72	69
query85	1373	293	273	273
query86	360	299	301	299
query87	4543	4376	4408	4376
query88	4076	2203	2168	2168
query89	390	291	291	291
query90	2247	183	181	181
query91	131	99	99	99
query92	64	48	49	48
query93	1076	526	532	526
query94	1084	287	271	271
query95	353	246	240	240
query96	618	286	283	283
query97	3265	3121	3174	3121
query98	211	201	198	198
query99	1581	1289	1299	1289
Total cold run time: 297497 ms
Total hot run time: 190392 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc5a183809529fd2eb07b31731896e2bc13ecfaa, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.06	0.06
query4	1.64	0.10	0.10
query5	0.50	0.49	0.49
query6	1.13	0.72	0.74
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.54
query10	0.54	0.54	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	2.81	2.79	2.73
query15	0.90	0.83	0.85
query16	0.39	0.39	0.39
query17	0.98	0.99	1.00
query18	0.20	0.20	0.20
query19	1.89	1.82	2.02
query20	0.01	0.01	0.01
query21	15.35	0.59	0.58
query22	2.59	1.95	2.03
query23	16.99	0.90	0.76
query24	3.43	1.18	1.38
query25	0.16	0.09	0.05
query26	0.72	0.13	0.14
query27	0.04	0.04	0.04
query28	9.96	1.11	1.08
query29	12.59	3.32	3.25
query30	0.25	0.06	0.06
query31	2.88	0.38	0.38
query32	3.28	0.45	0.46
query33	2.96	2.99	3.00
query34	17.04	4.45	4.49
query35	4.54	4.47	4.50
query36	0.66	0.49	0.52
query37	0.09	0.05	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.15	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.74 s
Total hot run time: 32.9 s

Copy link
Member

@zy-kkk zy-kkk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman merged commit 4948233 into apache:master Oct 17, 2024
morningman added a commit to morningman/doris that referenced this pull request Oct 18, 2024
…#41939)

Followup apache#38244
When setting `use_meta_cache=true`, the `include_database_list` and
`exclude_database_list`
will have no effect. This PR fix it.

Also fix a bug, related to apache#40479.
if `include_database_list` and `exclude_database_list` is set, and `use`
a database which is excluded,
Doris should return "database not found" error
morningman added a commit to morningman/doris that referenced this pull request Oct 21, 2024
…#41939)

Followup apache#38244
When setting `use_meta_cache=true`, the `include_database_list` and
`exclude_database_list`
will have no effect. This PR fix it.

Also fix a bug, related to apache#40479.
if `include_database_list` and `exclude_database_list` is set, and `use`
a database which is excluded,
Doris should return "database not found" error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants