Skip to content

[Fix](Row Store) all filter should match key columns condition #36400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
xiaokang merged 1 commit into from
Jun 18, 2024
Merged

[Fix](Row Store) all filter should match key columns condition #36400

xiaokang merged 1 commit into from
Jun 18, 2024

Conversation

@eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jun 17, 2024 •
edited
Loading

Queries like select * from tbl will pass LogicalResultSinkToShortCircuitPointQuery rule in the previous.
Introduced by #35823

Proposed changes

Issue Number: close #xxx

@eldenmoon
[Fix](Row Store) all filter should match key columns condition
216bf42 
query like `select * from tbl` will pass `LogicalResultSinkToShortCircuitPointQuery` rule in the previous
@doris-robot
Copy link

doris-robot commented Jun 17, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon
Copy link
Member Author

eldenmoon commented Jun 17, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jun 17, 2024

TPC-H: Total hot run time: 39888 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 216bf42bb2b5fe4e63f4c018792f693743250b4e, data reload: false

------ Round 1 ----------------------------------
q1	17812	4556	4433	4433
q2	2369	194	195	194
q3	10486	1167	1168	1167
q4	10194	770	789	770
q5	7471	2614	2630	2614
q6	221	140	137	137
q7	957	605	603	603
q8	9223	2070	2059	2059
q9	8904	6496	6499	6496
q10	8868	3697	3677	3677
q11	438	229	236	229
q12	449	240	224	224
q13	17758	2982	2995	2982
q14	269	218	229	218
q15	523	482	489	482
q16	520	378	378	378
q17	955	676	613	613
q18	8073	7532	7374	7374
q19	4820	1548	1503	1503
q20	646	303	329	303
q21	4890	3095	3187	3095
q22	384	337	341	337
Total cold run time: 116230 ms
Total hot run time: 39888 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4401	4245	4252	4245
q2	383	273	260	260
q3	2971	2676	2753	2676
q4	1859	1606	1626	1606
q5	5229	5225	5302	5225
q6	224	130	132	130
q7	2168	1729	1723	1723
q8	3219	3324	3325	3324
q9	8313	8389	8296	8296
q10	3885	3627	3681	3627
q11	582	491	495	491
q12	772	601	580	580
q13	17357	2989	3003	2989
q14	285	274	273	273
q15	533	478	466	466
q16	478	414	425	414
q17	1760	1466	1503	1466
q18	7678	7578	7332	7332
q19	1680	1505	1827	1505
q20	2023	1795	1792	1792
q21	4853	4729	4558	4558
q22	628	561	546	546
Total cold run time: 71281 ms
Total hot run time: 53524 ms

@doris-robot
Copy link

doris-robot commented Jun 17, 2024

TPC-DS: Total hot run time: 172695 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 216bf42bb2b5fe4e63f4c018792f693743250b4e, data reload: false

query1	928	385	366	366
query2	6465	2401	2190	2190
query3	6660	212	209	209
query4	19288	17606	17344	17344
query5	4139	494	458	458
query6	242	170	158	158
query7	4597	298	304	298
query8	318	299	296	296
query9	8507	2403	2372	2372
query10	618	301	278	278
query11	10347	10015	9963	9963
query12	128	82	85	82
query13	1638	361	357	357
query14	9569	7686	7638	7638
query15	281	191	190	190
query16	8053	279	269	269
query17	1895	555	543	543
query18	2069	285	286	285
query19	199	159	159	159
query20	96	82	84	82
query21	213	133	127	127
query22	4329	4062	4030	4030
query23	33616	32919	32926	32919
query24	11809	2881	2873	2873
query25	633	363	379	363
query26	1643	166	154	154
query27	2871	315	320	315
query28	6944	2051	2043	2043
query29	993	611	603	603
query30	261	154	148	148
query31	966	754	741	741
query32	93	53	60	53
query33	773	273	273	273
query34	970	483	468	468
query35	734	621	660	621
query36	1106	941	899	899
query37	164	72	69	69
query38	2870	2721	2751	2721
query39	842	792	790	790
query40	275	132	129	129
query41	59	52	56	52
query42	121	100	102	100
query43	592	541	546	541
query44	1276	724	749	724
query45	194	165	165	165
query46	1094	713	708	708
query47	1874	1752	1733	1733
query48	371	304	292	292
query49	1143	408	398	398
query50	792	384	392	384
query51	6817	6722	6589	6589
query52	105	93	95	93
query53	364	295	298	295
query54	910	437	442	437
query55	73	75	75	75
query56	283	294	265	265
query57	1117	1032	1051	1032
query58	253	233	264	233
query59	3309	3142	2969	2969
query60	297	277	289	277
query61	96	92	89	89
query62	671	445	454	445
query63	319	295	291	291
query64	9924	2306	1749	1749
query65	3131	3139	3099	3099
query66	1384	352	342	342
query67	15261	15098	14969	14969
query68	4609	534	537	534
query69	456	299	308	299
query70	1177	1088	1150	1088
query71	395	280	268	268
query72	7183	5256	5472	5256
query73	744	323	325	323
query74	6020	5487	5526	5487
query75	3430	2686	2673	2673
query76	2877	976	912	912
query77	458	308	299	299
query78	10286	9951	9742	9742
query79	2521	512	516	512
query80	2013	472	463	463
query81	600	220	226	220
query82	786	105	102	102
query83	369	167	168	167
query84	263	87	93	87
query85	1828	290	269	269
query86	487	303	275	275
query87	3240	3116	3104	3104
query88	3517	2349	2349	2349
query89	475	388	391	388
query90	1761	197	195	195
query91	134	99	100	99
query92	69	48	51	48
query93	1694	511	508	508
query94	1171	189	191	189
query95	408	313	322	313
query96	596	273	265	265
query97	3235	3016	3010	3010
query98	228	204	194	194
query99	1110	869	835	835
Total cold run time: 274112 ms
Total hot run time: 172695 ms

@doris-robot
Copy link

doris-robot commented Jun 17, 2024

ClickBench: Total hot run time: 30.09 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 216bf42bb2b5fe4e63f4c018792f693743250b4e, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.06
query5	0.47	0.50	0.50
query6	1.14	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.55	0.54	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.60
query14	0.76	0.77	0.78
query15	0.84	0.81	0.82
query16	0.37	0.37	0.35
query17	0.95	1.01	1.03
query18	0.23	0.23	0.27
query19	1.91	1.71	1.77
query20	0.02	0.01	0.01
query21	15.43	0.65	0.64
query22	4.19	7.57	1.58
query23	18.26	1.36	1.14
query24	2.20	0.21	0.22
query25	0.15	0.08	0.07
query26	0.25	0.18	0.17
query27	0.08	0.07	0.08
query28	13.20	1.03	1.00
query29	12.64	3.28	3.32
query30	0.26	0.07	0.06
query31	2.86	0.40	0.39
query32	3.28	0.48	0.47
query33	2.86	2.92	2.89
query34	17.17	4.47	4.39
query35	4.47	4.48	4.49
query36	0.66	0.46	0.47
query37	0.18	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.03
query40	0.18	0.15	0.15
query41	0.09	0.05	0.05
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.5 s
Total hot run time: 30.09 s

zhannngchen
zhannngchen approved these changes Jun 17, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 17, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 17, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 17, 2024

PR approved by anyone and no changes requested.

xiaokang
xiaokang approved these changes Jun 18, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang merged commit b4b7df5 into apache:master Jun 18, 2024
@xiaokang xiaokang added the p0_b label Jun 18, 2024
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jun 18, 2024
@eldenmoon
[Fix](Row Store) all filter should match key columns condition (apach…
29eb35b 
…e#36400)

Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by apache#35823
eldenmoon added a commit that referenced this pull request Jun 19, 2024
@eldenmoon
[Fix](Row Store) all filter should match key columns condition (#36400)…
38d750a 
… (#36443)

Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by #35823
dataroaring pushed a commit that referenced this pull request Jun 21, 2024
@eldenmoon @dataroaring
[Fix](Row Store) all filter should match key columns condition (#36400)
654cead 
Queries like `select * from tbl` will pass
`LogicalResultSinkToShortCircuitPointQuery` rule in the previous.
Introduced by #35823
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokang xiaokang xiaokang approved these changes

@zhannngchen zhannngchen zhannngchen approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/2.1.5-merged dev/3.0.0-merged p0_b reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@eldenmoon @doris-robot @xiaokang @zhannngchen @morningman @dataroaring