Skip to content

[Feature](Point Query) fully support in nereids #35823

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eldenmoon merged 5 commits into from
Jun 13, 2024
Merged

[Feature](Point Query) fully support in nereids #35823

eldenmoon merged 5 commits into from
Jun 13, 2024

Conversation

@eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jun 3, 2024 •
edited
Loading

Proposed changes

step2: support short circuit query and short circuit(full) prepared

step1 PR: #35318
original implementation in legacy planner: #15491
Issue Number: close #xxx

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 3, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-H: Total hot run time: 40576 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6600f4c2beeb29948169fe0a441a1e1179526397, data reload: false

------ Round 1 ----------------------------------
q1	17615	4727	4227	4227
q2	2025	185	191	185
q3	10455	1255	1139	1139
q4	10182	861	757	757
q5	7499	2727	2697	2697
q6	231	133	138	133
q7	978	608	629	608
q8	9221	2119	2084	2084
q9	9282	6649	6630	6630
q10	9227	3917	3894	3894
q11	479	269	244	244
q12	467	238	229	229
q13	18884	3223	3200	3200
q14	267	216	219	216
q15	508	491	481	481
q16	506	400	369	369
q17	972	656	605	605
q18	8368	7964	7741	7741
q19	4671	1348	1280	1280
q20	668	317	298	298
q21	4996	3215	4023	3215
q22	403	344	350	344
Total cold run time: 117904 ms
Total hot run time: 40576 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4731	4478	4497	4478
q2	377	265	268	265
q3	3125	2918	2922	2918
q4	1926	1632	1548	1548
q5	5380	5476	5489	5476
q6	230	128	132	128
q7	2166	1811	1798	1798
q8	3227	3320	3373	3320
q9	8602	8559	8590	8559
q10	4078	3891	3850	3850
q11	591	487	494	487
q12	778	614	612	612
q13	15827	3047	3157	3047
q14	298	261	272	261
q15	509	468	482	468
q16	487	423	421	421
q17	1767	1473	1478	1473
q18	8061	7751	7326	7326
q19	1739	1494	1530	1494
q20	3016	1783	1786	1783
q21	8097	4824	4758	4758
q22	595	534	551	534
Total cold run time: 75607 ms
Total hot run time: 55004 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-DS: Total hot run time: 169826 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6600f4c2beeb29948169fe0a441a1e1179526397, data reload: false

query1	941	371	364	364
query2	6451	2406	2297	2297
query3	6647	200	205	200
query4	20970	17406	17103	17103
query5	4105	465	455	455
query6	256	168	155	155
query7	4579	302	308	302
query8	333	284	287	284
query9	8544	2409	2359	2359
query10	439	304	272	272
query11	10601	9959	10024	9959
query12	137	89	87	87
query13	1639	366	366	366
query14	9509	7998	6994	6994
query15	242	186	186	186
query16	7813	280	261	261
query17	1734	535	533	533
query18	1956	265	267	265
query19	206	151	155	151
query20	98	84	85	84
query21	207	137	123	123
query22	4180	4095	4021	4021
query23	33605	33029	32859	32859
query24	10867	2962	2874	2874
query25	604	350	370	350
query26	1105	159	164	159
query27	2308	313	325	313
query28	6696	2049	2099	2049
query29	868	604	605	604
query30	282	150	160	150
query31	950	743	736	736
query32	92	51	59	51
query33	767	291	272	272
query34	944	477	472	472
query35	724	615	590	590
query36	1079	941	903	903
query37	134	67	69	67
query38	2886	2765	2698	2698
query39	868	824	783	783
query40	213	129	124	124
query41	53	53	54	53
query42	118	94	97	94
query43	589	568	540	540
query44	1234	760	771	760
query45	200	171	167	167
query46	1082	729	736	729
query47	1891	1737	1752	1737
query48	385	301	308	301
query49	1003	419	415	415
query50	782	400	399	399
query51	6753	6785	6792	6785
query52	106	88	93	88
query53	356	293	294	293
query54	882	466	449	449
query55	76	73	72	72
query56	277	249	261	249
query57	1138	1033	1029	1029
query58	258	245	240	240
query59	3597	3487	3344	3344
query60	300	268	301	268
query61	91	88	86	86
query62	632	448	436	436
query63	322	294	289	289
query64	8836	2201	1751	1751
query65	3185	3116	3293	3116
query66	809	323	326	323
query67	15421	14894	14873	14873
query68	4488	540	534	534
query69	456	295	304	295
query70	1151	1132	1139	1132
query71	404	289	292	289
query72	7042	5579	2722	2722
query73	727	327	322	322
query74	5900	5506	5502	5502
query75	3340	2699	2660	2660
query76	2295	914	972	914
query77	442	312	291	291
query78	10224	9716	9704	9704
query79	1926	524	519	519
query80	825	456	486	456
query81	591	219	223	219
query82	900	105	100	100
query83	256	176	176	176
query84	258	90	99	90
query85	1316	268	273	268
query86	456	310	289	289
query87	3296	3087	3167	3087
query88	3921	2446	2454	2446
query89	466	399	388	388
query90	1762	185	187	185
query91	125	102	94	94
query92	58	47	49	47
query93	2056	515	504	504
query94	1218	190	184	184
query95	401	318	312	312
query96	596	280	272	272
query97	3183	3025	3007	3007
query98	242	216	216	216
query99	1343	847	841	841
Total cold run time: 268619 ms
Total hot run time: 169826 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

ClickBench: Total hot run time: 30.78 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6600f4c2beeb29948169fe0a441a1e1179526397, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.06	0.07
query5	0.49	0.49	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.05
query9	0.54	0.49	0.48
query10	0.53	0.55	0.54
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.76	0.78	0.78
query15	0.83	0.81	0.80
query16	0.37	0.37	0.38
query17	0.97	1.02	0.99
query18	0.23	0.24	0.24
query19	1.80	1.73	1.83
query20	0.02	0.01	0.01
query21	15.43	0.73	0.70
query22	4.77	6.68	2.00
query23	18.42	1.43	1.30
query24	1.38	0.32	0.24
query25	0.14	0.09	0.08
query26	0.26	0.18	0.17
query27	0.07	0.08	0.09
query28	13.51	1.03	1.00
query29	13.10	3.36	3.26
query30	0.24	0.06	0.05
query31	2.86	0.39	0.38
query32	3.28	0.47	0.46
query33	2.93	2.86	2.90
query34	17.14	4.40	4.42
query35	4.52	4.60	4.55
query36	0.67	0.45	0.46
query37	0.18	0.16	0.16
query38	0.16	0.14	0.15
query39	0.04	0.03	0.04
query40	0.19	0.14	0.14
query41	0.10	0.05	0.04
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.13 s
Total hot run time: 30.78 s

@morrySnow
Copy link
Contributor

morrySnow commented Jun 4, 2024

  1. add the step 1 PR id in desc.
  2. add legacy planner's point query pr id in description or write some breif desc about point query.

@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 4, 2024

  1. add the step 1 PR id in desc.
  2. add legacy planner's point query pr id in description or write some breif desc about point query.

done

@eldenmoon eldenmoon requested a review from xiaokang June 5, 2024 02:56
xiaokang
xiaokang reviewed Jun 9, 2024
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java Outdated
public static final String MAX_INSTANCE_NUM = "max_instance_num";
public static final String ENABLE_INSERT_STRICT = "enable_insert_strict";
public static final String ENABLE_SPILLING = "enable_spilling";
public static final String ENABLE_SHORT_CIRCUIT_QUERY = "enable_short_circuit_query";
Copy link
Contributor

@xiaokang xiaokang Jun 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest a more specific name enable_short_circuit_point_query

fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java Outdated

@Override
public void exec() throws Exception {
// Do nothing
Copy link
Contributor

@xiaokang xiaokang Jun 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do nothing? Add comment to explain.

@eldenmoon eldenmoon force-pushed the nereids-short-circuit branch from 6600f4c to 07b73bc Compare June 11, 2024 06:46
@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 11, 2024

run buildall

xiaokang
xiaokang previously approved these changes Jun 11, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 11, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 11, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 11, 2024

PR approved by anyone and no changes requested.

@eldenmoon eldenmoon requested a review from morrySnow June 11, 2024 07:06
@doris-robot
Copy link

doris-robot commented Jun 11, 2024

TPC-H: Total hot run time: 39579 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 07b73bc0a7ce7d111308957d6491b44e7f9d6d78, data reload: false

------ Round 1 ----------------------------------
q1	17631	4370	4283	4283
q2	2025	189	188	188
q3	10473	1129	1099	1099
q4	10194	795	772	772
q5	7485	2751	2603	2603
q6	220	137	134	134
q7	958	601	585	585
q8	9215	2045	2021	2021
q9	8818	6458	6440	6440
q10	8938	3839	3758	3758
q11	465	237	236	236
q12	454	232	219	219
q13	17821	2937	2970	2937
q14	263	220	227	220
q15	519	480	486	480
q16	531	374	370	370
q17	950	656	725	656
q18	7950	7342	7343	7342
q19	7104	1413	1514	1413
q20	671	319	324	319
q21	4959	3812	3177	3177
q22	376	327	329	327
Total cold run time: 118020 ms
Total hot run time: 39579 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4364	4234	4191	4191
q2	368	262	268	262
q3	2959	2940	2822	2822
q4	1988	1702	1758	1702
q5	5545	5475	5440	5440
q6	221	129	129	129
q7	2213	1903	1826	1826
q8	3307	3417	3406	3406
q9	8668	8659	8768	8659
q10	4110	3851	3710	3710
q11	613	479	481	479
q12	844	644	623	623
q13	15895	3156	3158	3156
q14	285	273	277	273
q15	516	485	479	479
q16	481	418	429	418
q17	1815	1504	1526	1504
q18	8036	7815	7732	7732
q19	1781	1604	1618	1604
q20	2367	1911	1815	1815
q21	5171	5010	4972	4972
q22	622	515	517	515
Total cold run time: 72169 ms
Total hot run time: 55717 ms

@doris-robot
Copy link

doris-robot commented Jun 11, 2024

TPC-DS: Total hot run time: 172514 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 07b73bc0a7ce7d111308957d6491b44e7f9d6d78, data reload: false

query1	921	390	374	374
query2	6424	2556	2383	2383
query3	6636	205	207	205
query4	21182	17257	17237	17237
query5	3630	485	476	476
query6	235	152	158	152
query7	4583	289	290	289
query8	319	286	311	286
query9	8441	2369	2338	2338
query10	547	288	283	283
query11	10351	9895	9859	9859
query12	119	86	82	82
query13	1647	359	360	359
query14	10062	6421	7666	6421
query15	230	189	184	184
query16	7681	270	258	258
query17	1602	525	507	507
query18	1729	274	274	274
query19	203	156	176	156
query20	91	84	82	82
query21	209	127	127	127
query22	4329	4175	4117	4117
query23	33640	33682	33773	33682
query24	11147	2875	2874	2874
query25	619	382	363	363
query26	724	151	157	151
query27	2299	333	331	331
query28	5748	2129	2101	2101
query29	919	635	650	635
query30	231	156	174	156
query31	936	749	786	749
query32	84	52	55	52
query33	774	279	271	271
query34	1080	482	476	476
query35	728	651	621	621
query36	1135	1004	1005	1004
query37	143	76	70	70
query38	2955	2835	2823	2823
query39	893	843	847	843
query40	205	130	133	130
query41	55	51	53	51
query42	111	103	100	100
query43	601	559	559	559
query44	1181	714	730	714
query45	196	160	165	160
query46	1090	750	740	740
query47	1845	1784	1799	1784
query48	372	290	284	284
query49	838	395	395	395
query50	775	385	386	385
query51	6866	6735	6659	6659
query52	106	90	100	90
query53	353	289	286	286
query54	925	433	428	428
query55	73	74	70	70
query56	281	244	279	244
query57	1122	1038	1060	1038
query58	287	244	252	244
query59	3445	3046	3113	3046
query60	305	275	295	275
query61	112	106	102	102
query62	629	449	436	436
query63	329	287	294	287
query64	8568	2208	1729	1729
query65	3194	3089	3100	3089
query66	749	343	329	329
query67	15427	15170	14846	14846
query68	4535	542	568	542
query69	513	291	298	291
query70	1158	1064	1145	1064
query71	446	269	273	269
query72	8003	5475	5380	5380
query73	763	321	327	321
query74	5945	5488	5452	5452
query75	3397	2667	2640	2640
query76	2464	965	888	888
query77	677	294	291	291
query78	10309	9797	9708	9708
query79	2865	507	515	507
query80	1396	456	453	453
query81	547	213	217	213
query82	745	103	99	99
query83	344	164	160	160
query84	275	88	89	88
query85	1918	266	272	266
query86	480	328	305	305
query87	3311	3077	3091	3077
query88	3795	2352	2356	2352
query89	459	372	390	372
query90	1802	188	183	183
query91	124	93	95	93
query92	60	118	48	48
query93	3208	496	494	494
query94	1142	182	181	181
query95	384	306	304	304
query96	596	273	277	273
query97	3207	3043	3027	3027
query98	225	201	191	191
query99	1135	877	831	831
Total cold run time: 271494 ms
Total hot run time: 172514 ms

@doris-robot
Copy link

doris-robot commented Jun 11, 2024

ClickBench: Total hot run time: 30.4 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 07b73bc0a7ce7d111308957d6491b44e7f9d6d78, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.05	0.05
query4	1.68	0.06	0.07
query5	0.50	0.49	0.48
query6	1.13	0.72	0.72
query7	0.02	0.02	0.02
query8	0.06	0.04	0.05
query9	0.55	0.51	0.47
query10	0.56	0.55	0.53
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.60
query14	0.79	0.80	0.79
query15	0.82	0.82	0.81
query16	0.37	0.36	0.36
query17	0.97	0.96	1.00
query18	0.22	0.27	0.24
query19	1.76	1.72	1.73
query20	0.02	0.01	0.01
query21	15.40	0.66	0.65
query22	4.70	6.69	2.00
query23	18.30	1.30	1.14
query24	2.09	0.22	0.22
query25	0.15	0.07	0.09
query26	0.27	0.18	0.18
query27	0.08	0.07	0.08
query28	13.24	1.01	0.99
query29	12.69	3.32	3.24
query30	0.26	0.07	0.06
query31	2.87	0.40	0.38
query32	3.27	0.47	0.47
query33	2.87	2.83	2.94
query34	17.21	4.42	4.46
query35	4.52	4.50	4.50
query36	0.66	0.45	0.47
query37	0.18	0.14	0.14
query38	0.16	0.15	0.14
query39	0.05	0.04	0.04
query40	0.18	0.14	0.14
query41	0.10	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.03 s
Total hot run time: 30.4 s

morrySnow
morrySnow reviewed Jun 12, 2024
View reviewed changes
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 12, 2024
@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 12, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-H: Total hot run time: 39916 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3bf111d56937ec722027719fab2297aa02238f3f, data reload: false

------ Round 1 ----------------------------------
q1	18588	4507	4417	4417
q2	2653	196	197	196
q3	12112	1127	1073	1073
q4	10874	780	783	780
q5	8093	3170	2640	2640
q6	219	134	138	134
q7	953	604	602	602
q8	9219	2074	2084	2074
q9	8808	6479	6472	6472
q10	8959	3758	3696	3696
q11	455	243	234	234
q12	394	229	216	216
q13	18917	2979	2960	2960
q14	253	221	216	216
q15	513	476	482	476
q16	508	383	380	380
q17	978	635	743	635
q18	8033	7656	7386	7386
q19	7596	1555	1539	1539
q20	660	321	341	321
q21	5040	3137	3251	3137
q22	380	332	332	332
Total cold run time: 124205 ms
Total hot run time: 39916 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4340	4270	4218	4218
q2	375	270	282	270
q3	2986	2742	2725	2725
q4	1855	1572	1595	1572
q5	5245	5306	5273	5273
q6	210	123	128	123
q7	2150	1692	1732	1692
q8	3210	3317	3321	3317
q9	8336	8386	8327	8327
q10	3927	3670	3667	3667
q11	595	481	471	471
q12	763	603	611	603
q13	17683	2944	2975	2944
q14	296	266	258	258
q15	519	463	475	463
q16	461	415	426	415
q17	1747	1479	1502	1479
q18	7622	7478	7348	7348
q19	1690	1501	1537	1501
q20	1973	1788	1763	1763
q21	4999	4698	4830	4698
q22	583	554	560	554
Total cold run time: 71565 ms
Total hot run time: 53681 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-DS: Total hot run time: 172266 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3bf111d56937ec722027719fab2297aa02238f3f, data reload: false

query1	935	388	367	367
query2	6440	2564	2328	2328
query3	6662	212	215	212
query4	18946	17288	17216	17216
query5	4148	463	459	459
query6	256	171	151	151
query7	4596	305	308	305
query8	348	306	302	302
query9	8492	2361	2325	2325
query10	609	311	290	290
query11	10600	10047	10130	10047
query12	138	83	88	83
query13	1641	376	367	367
query14	9448	7343	6962	6962
query15	225	187	185	185
query16	7831	269	266	266
query17	1917	548	537	537
query18	1962	271	298	271
query19	196	157	154	154
query20	93	84	80	80
query21	214	129	123	123
query22	4253	3989	4126	3989
query23	33629	33017	33088	33017
query24	12024	2814	2785	2785
query25	659	354	361	354
query26	1819	153	150	150
query27	2989	324	315	315
query28	7550	2038	2017	2017
query29	1199	640	642	640
query30	274	152	154	152
query31	948	734	780	734
query32	101	60	57	57
query33	787	305	296	296
query34	931	483	463	463
query35	751	622	638	622
query36	1105	970	955	955
query37	212	75	72	72
query38	2865	2782	2751	2751
query39	856	792	836	792
query40	275	127	123	123
query41	54	53	51	51
query42	120	101	100	100
query43	593	557	540	540
query44	1174	732	745	732
query45	194	167	168	167
query46	1096	704	747	704
query47	1850	1745	1743	1743
query48	385	295	299	295
query49	1190	407	402	402
query50	777	378	397	378
query51	6852	6714	6619	6619
query52	101	95	99	95
query53	358	297	283	283
query54	1017	461	439	439
query55	75	75	74	74
query56	316	252	270	252
query57	1113	1040	1044	1040
query58	253	238	244	238
query59	3713	3330	3139	3139
query60	286	276	273	273
query61	91	107	96	96
query62	642	452	450	450
query63	328	289	293	289
query64	9887	2237	1719	1719
query65	3177	3125	3099	3099
query66	1383	345	336	336
query67	15438	15001	14862	14862
query68	4759	552	542	542
query69	597	449	375	375
query70	1116	1056	1118	1056
query71	417	274	282	274
query72	7425	5141	5524	5141
query73	805	330	325	325
query74	5948	5556	5509	5509
query75	3659	2687	2667	2667
query76	3229	958	966	958
query77	659	309	304	304
query78	10428	9786	9735	9735
query79	2681	508	516	508
query80	1362	475	453	453
query81	563	218	219	218
query82	768	113	99	99
query83	203	168	173	168
query84	272	89	87	87
query85	1435	297	277	277
query86	460	295	310	295
query87	3328	3080	3065	3065
query88	3716	2457	2447	2447
query89	476	384	394	384
query90	1823	200	194	194
query91	148	104	102	102
query92	60	47	54	47
query93	3249	523	498	498
query94	1158	188	189	188
query95	391	309	318	309
query96	588	272	273	272
query97	3223	3035	3024	3024
query98	218	203	189	189
query99	1460	842	827	827
Total cold run time: 278011 ms
Total hot run time: 172266 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

ClickBench: Total hot run time: 30.19 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3bf111d56937ec722027719fab2297aa02238f3f, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.06	0.07
query5	0.51	0.48	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.55	0.54
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	0.79	0.81	0.76
query15	0.83	0.81	0.82
query16	0.35	0.35	0.37
query17	1.01	0.96	0.98
query18	0.22	0.24	0.25
query19	1.79	1.69	1.78
query20	0.02	0.01	0.01
query21	15.43	0.65	0.64
query22	3.80	8.44	1.61
query23	18.32	1.42	1.30
query24	2.16	0.23	0.21
query25	0.15	0.07	0.08
query26	0.27	0.18	0.17
query27	0.08	0.09	0.09
query28	13.20	1.00	1.00
query29	12.62	3.22	3.25
query30	0.25	0.07	0.06
query31	2.85	0.38	0.38
query32	3.29	0.47	0.47
query33	2.92	2.88	2.90
query34	17.10	4.42	4.41
query35	4.49	4.51	4.56
query36	0.66	0.49	0.48
query37	0.18	0.16	0.15
query38	0.15	0.14	0.15
query39	0.04	0.04	0.04
query40	0.17	0.15	0.14
query41	0.09	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.08 s
Total hot run time: 30.19 s

@eldenmoon eldenmoon dismissed stale reviews from qidaye and xiaokang via 1fbb929 June 12, 2024 12:46
@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 12, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 12, 2024
@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-H: Total hot run time: 39787 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1fbb929468fbfc38cc4bff12d8ca814a5d23cdcb, data reload: false

------ Round 1 ----------------------------------
q1	17615	4515	4286	4286
q2	2045	189	190	189
q3	10552	1142	1078	1078
q4	10276	766	728	728
q5	7541	2691	2675	2675
q6	231	139	139	139
q7	982	626	614	614
q8	9263	2072	2112	2072
q9	9389	6536	6440	6440
q10	8891	3675	3728	3675
q11	453	239	238	238
q12	508	243	227	227
q13	17771	2974	2961	2961
q14	282	220	221	220
q15	531	480	475	475
q16	491	375	375	375
q17	962	638	672	638
q18	7961	7466	7393	7393
q19	1742	1532	1504	1504
q20	645	306	315	306
q21	5080	3220	3928	3220
q22	384	345	334	334
Total cold run time: 113595 ms
Total hot run time: 39787 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4345	4243	4251	4243
q2	369	258	263	258
q3	2965	2725	2754	2725
q4	1871	1652	1602	1602
q5	5229	5263	5246	5246
q6	215	127	130	127
q7	2110	1753	1677	1677
q8	3218	3304	3296	3296
q9	8322	8248	8308	8248
q10	3883	3617	3675	3617
q11	585	500	490	490
q12	761	625	607	607
q13	17518	2977	2998	2977
q14	298	269	264	264
q15	524	474	464	464
q16	462	412	418	412
q17	1759	1487	1459	1459
q18	7642	7495	7275	7275
q19	1704	1668	1566	1566
q20	1990	1769	1764	1764
q21	4833	4769	4670	4670
q22	572	582	549	549
Total cold run time: 71175 ms
Total hot run time: 53536 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

TPC-DS: Total hot run time: 173236 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1fbb929468fbfc38cc4bff12d8ca814a5d23cdcb, data reload: false

query1	938	384	360	360
query2	6446	2427	2470	2427
query3	6662	209	216	209
query4	19675	17551	17123	17123
query5	4132	465	445	445
query6	253	162	162	162
query7	4593	294	289	289
query8	330	293	276	276
query9	8548	2378	2345	2345
query10	630	301	291	291
query11	10742	10219	10040	10040
query12	141	93	86	86
query13	1659	361	374	361
query14	10139	7539	7576	7539
query15	239	187	184	184
query16	7666	282	258	258
query17	1875	529	549	529
query18	1887	278	269	269
query19	214	162	159	159
query20	92	80	80	80
query21	215	134	132	132
query22	4445	4124	4001	4001
query23	33832	33066	32959	32959
query24	11272	2984	2958	2958
query25	665	357	358	357
query26	1645	153	163	153
query27	2924	306	319	306
query28	7271	2007	2011	2007
query29	1027	643	598	598
query30	277	150	150	150
query31	951	740	754	740
query32	96	55	57	55
query33	823	282	269	269
query34	957	475	450	450
query35	760	622	611	611
query36	1146	969	935	935
query37	161	70	73	70
query38	2922	2719	2727	2719
query39	878	786	822	786
query40	286	123	129	123
query41	57	51	50	50
query42	119	94	103	94
query43	603	545	570	545
query44	1204	729	740	729
query45	192	169	164	164
query46	1199	751	743	743
query47	1896	1797	1795	1795
query48	367	306	295	295
query49	1148	401	400	400
query50	827	383	373	373
query51	6878	6712	6674	6674
query52	112	92	96	92
query53	398	293	287	287
query54	998	461	444	444
query55	77	75	75	75
query56	300	280	280	280
query57	1173	1053	1095	1053
query58	252	243	257	243
query59	3465	3217	3224	3217
query60	303	263	283	263
query61	96	89	93	89
query62	661	429	461	429
query63	326	295	289	289
query64	10125	2333	1720	1720
query65	3241	3127	3083	3083
query66	1374	336	339	336
query67	15426	15001	15007	15001
query68	4536	534	538	534
query69	465	292	303	292
query70	1207	1125	1127	1125
query71	388	271	260	260
query72	7135	5903	5314	5314
query73	804	325	320	320
query74	6013	5427	5482	5427
query75	3464	2655	2679	2655
query76	2372	986	989	986
query77	447	291	282	282
query78	10473	9802	9826	9802
query79	2564	515	513	513
query80	1178	463	457	457
query81	586	216	216	216
query82	817	106	99	99
query83	234	174	170	170
query84	245	84	86	84
query85	1547	291	272	272
query86	477	305	342	305
query87	3288	3059	3066	3059
query88	4115	2462	2439	2439
query89	501	396	382	382
query90	1789	188	191	188
query91	133	99	98	98
query92	61	50	50	50
query93	1859	514	503	503
query94	1190	190	183	183
query95	401	311	308	308
query96	644	270	271	270
query97	3322	3033	3041	3033
query98	291	205	199	199
query99	1233	847	837	837
Total cold run time: 276010 ms
Total hot run time: 173236 ms

@doris-robot
Copy link

doris-robot commented Jun 12, 2024

ClickBench: Total hot run time: 30.93 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1fbb929468fbfc38cc4bff12d8ca814a5d23cdcb, data reload: false

query1	0.03	0.03	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.51	0.48	0.50
query6	1.12	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.49	0.49
query10	0.54	0.54	0.54
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.58
query14	0.75	0.78	0.78
query15	0.84	0.81	0.82
query16	0.36	0.37	0.36
query17	0.96	0.97	0.97
query18	0.21	0.24	0.25
query19	1.88	1.69	1.84
query20	0.01	0.01	0.01
query21	15.42	0.66	0.65
query22	4.33	6.95	2.32
query23	18.27	1.44	1.24
query24	2.09	0.22	0.22
query25	0.16	0.07	0.09
query26	0.26	0.17	0.17
query27	0.08	0.07	0.08
query28	13.32	1.00	0.99
query29	12.59	3.36	3.35
query30	0.26	0.06	0.06
query31	2.86	0.39	0.38
query32	3.28	0.46	0.48
query33	2.85	2.87	2.96
query34	17.28	4.42	4.43
query35	4.50	4.49	4.49
query36	0.66	0.46	0.47
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.21	0.15	0.14
query41	0.10	0.05	0.04
query42	0.05	0.04	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.69 s
Total hot run time: 30.93 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 12, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 12, 2024

PR approved by at least one committer and no changes requested.

xiaokang pushed a commit that referenced this pull request Jun 13, 2024
@eldenmoon
[Feature](Point Query) fully support in nereids #35823 (#36205)
226775f
@eldenmoon eldenmoon merged commit 2f517a9 into apache:master Jun 13, 2024
@eldenmoon eldenmoon deleted the nereids-short-circuit branch June 13, 2024 02:14
dataroaring pushed a commit that referenced this pull request Jun 13, 2024
@eldenmoon @dataroaring
[Feature](Point Query) fully support in nereids (#35823)
166ef6d 
step2: support short circuit query and short circuit(full) prepared

step1 PR: #35318
original implementation in legacy planner: #15491
@eldenmoon eldenmoon mentioned this pull request Jun 17, 2024
Merged
xiaokang pushed a commit that referenced this pull request Jun 18, 2024
@eldenmoon
[Fix](Row Store) all filter should match key columns condition (#36400)
b4b7df5 
Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by #35823
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jun 18, 2024
@eldenmoon
[Fix](Row Store) all filter should match key columns condition (apach…
29eb35b 
…e#36400)

Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by apache#35823
@eldenmoon eldenmoon mentioned this pull request Jun 18, 2024
Merged
eldenmoon added a commit that referenced this pull request Jun 19, 2024
@eldenmoon
[Fix](Row Store) all filter should match key columns condition (#36400)…
38d750a 
… (#36443)

Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by #35823
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jun 19, 2024
@morrySnow
[fix](Nereids) should check it is Slot before check it is DELETE_SIGN
bc6ef41 
introduced by apache#35823
@morrySnow morrySnow mentioned this pull request Jun 19, 2024
Merged
starocean999 pushed a commit that referenced this pull request Jun 20, 2024
@morrySnow
[fix](Nereids) should check it is Slot before check it is DELETE_SIGN (…
cdcb02a 
…#36564)

introduced by #35823
iszhangpch pushed a commit to iszhangpch/doris-p that referenced this pull request Jun 21, 2024
@morrySnow
[fix](Nereids) should check it is Slot before check it is DELETE_SIGN (…
5c1924e 
…apache#36564)

introduced by apache#35823
dataroaring pushed a commit that referenced this pull request Jun 21, 2024
@eldenmoon @dataroaring
[Fix](Row Store) all filter should match key columns condition (#36400)
654cead 
Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by #35823
dataroaring pushed a commit that referenced this pull request Jun 21, 2024
@morrySnow @dataroaring
[fix](Nereids) should check it is Slot before check it is DELETE_SIGN (…
32c8dec 
…#36564)

introduced by #35823
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokang xiaokang xiaokang approved these changes

@qidaye qidaye qidaye approved these changes

@morrySnow morrySnow morrySnow left review comments

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@eldenmoon @doris-robot @morrySnow @xiaokang @qidaye @dataroaring