Skip to content

[fix](nereids) fix prune partitions wrong #48034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
924060929 merged 1 commit into from
Feb 19, 2025
Merged

[fix](nereids) fix prune partitions wrong #48034

924060929 merged 1 commit into from
Feb 19, 2025

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Feb 18, 2025 •
edited
Loading

What problem does this PR solve?

fix prune partitions wrong, introduced by #44574

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 18, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

924060929 commented Feb 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Feb 18, 2025

TPC-H: Total hot run time: 31343 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6d79a08d92cc7b33cf0c17f12152e8a6df21662e, data reload: false

------ Round 1 ----------------------------------
q1	17609	5292	5103	5103
q2	2046	313	179	179
q3	10391	1325	712	712
q4	10212	1016	533	533
q5	7534	2397	2324	2324
q6	193	170	132	132
q7	908	766	611	611
q8	9292	1321	1046	1046
q9	5508	4535	4792	4535
q10	6809	2313	1891	1891
q11	479	274	260	260
q12	347	360	221	221
q13	17760	3651	3059	3059
q14	223	232	203	203
q15	493	463	464	463
q16	617	620	585	585
q17	548	882	331	331
q18	6817	6274	6134	6134
q19	1204	953	545	545
q20	310	329	204	204
q21	2727	2214	1964	1964
q22	370	335	308	308
Total cold run time: 102397 ms
Total hot run time: 31343 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5212	5220	5240	5220
q2	233	337	242	242
q3	2215	2696	2284	2284
q4	1476	1843	1394	1394
q5	4247	4156	4169	4156
q6	207	166	124	124
q7	1859	1808	1729	1729
q8	2607	2710	2651	2651
q9	7216	7047	7118	7047
q10	2990	3211	2729	2729
q11	602	505	476	476
q12	722	801	641	641
q13	3547	3829	3221	3221
q14	297	293	294	293
q15	513	464	466	464
q16	632	697	634	634
q17	1134	1601	1344	1344
q18	7495	7439	7277	7277
q19	786	809	861	809
q20	1950	2043	1871	1871
q21	5422	4977	4713	4713
q22	640	583	562	562
Total cold run time: 52002 ms
Total hot run time: 49881 ms

@doris-robot
Copy link

doris-robot commented Feb 18, 2025

TPC-DS: Total hot run time: 183583 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6d79a08d92cc7b33cf0c17f12152e8a6df21662e, data reload: false

query1	986	380	383	380
query2	6527	1938	1903	1903
query3	6782	211	210	210
query4	26065	23494	23337	23337
query5	4371	654	485	485
query6	307	191	192	191
query7	4598	502	309	309
query8	317	249	236	236
query9	8643	2521	2534	2521
query10	447	317	250	250
query11	15607	15091	14934	14934
query12	160	108	103	103
query13	1668	530	404	404
query14	10281	6110	6227	6110
query15	210	196	175	175
query16	7565	641	488	488
query17	1204	711	556	556
query18	1958	408	312	312
query19	192	186	157	157
query20	120	114	123	114
query21	212	129	103	103
query22	4139	4348	4509	4348
query23	34387	33394	33045	33045
query24	7689	2337	2361	2337
query25	524	445	391	391
query26	1230	268	153	153
query27	2173	498	326	326
query28	3972	2406	2366	2366
query29	697	539	420	420
query30	226	185	169	169
query31	948	867	790	790
query32	73	63	59	59
query33	556	342	297	297
query34	786	858	488	488
query35	784	799	747	747
query36	952	961	886	886
query37	113	100	71	71
query38	4054	4126	4233	4126
query39	1443	1385	1377	1377
query40	202	119	101	101
query41	52	54	49	49
query42	128	101	105	101
query43	492	518	487	487
query44	1265	796	804	796
query45	175	168	158	158
query46	848	1030	657	657
query47	1763	1809	1741	1741
query48	369	401	298	298
query49	777	514	401	401
query50	692	751	403	403
query51	4187	4210	4097	4097
query52	108	105	95	95
query53	222	257	181	181
query54	481	481	406	406
query55	82	83	79	79
query56	259	279	269	269
query57	1131	1137	1076	1076
query58	243	229	237	229
query59	2607	2646	2678	2646
query60	279	264	249	249
query61	121	116	136	116
query62	784	713	668	668
query63	216	188	188	188
query64	4289	1024	655	655
query65	3247	3103	3129	3103
query66	1087	406	291	291
query67	16042	15394	15343	15343
query68	5750	794	521	521
query69	510	287	278	278
query70	1187	1084	1115	1084
query71	413	298	261	261
query72	5997	3646	3812	3646
query73	745	755	358	358
query74	8926	9202	8767	8767
query75	3119	3142	2735	2735
query76	3270	1174	730	730
query77	467	382	286	286
query78	10023	10350	9213	9213
query79	936	902	596	596
query80	1181	541	481	481
query81	542	277	299	277
query82	429	128	92	92
query83	270	174	156	156
query84	242	93	77	77
query85	782	348	310	310
query86	384	299	286	286
query87	4418	4488	4452	4452
query88	2848	2226	2193	2193
query89	384	321	279	279
query90	1704	195	196	195
query91	136	135	109	109
query92	60	62	55	55
query93	1119	1016	599	599
query94	661	412	287	287
query95	360	260	257	257
query96	477	554	269	269
query97	2709	2868	2751	2751
query98	224	208	195	195
query99	1313	1387	1231	1231
Total cold run time: 266700 ms
Total hot run time: 183583 ms

@doris-robot
Copy link

doris-robot commented Feb 18, 2025

ClickBench: Total hot run time: 30.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6d79a08d92cc7b33cf0c17f12152e8a6df21662e, data reload: false

query1	0.03	0.04	0.05
query2	0.07	0.04	0.04
query3	0.24	0.06	0.07
query4	1.62	0.11	0.10
query5	0.41	0.40	0.40
query6	1.17	0.66	0.65
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.51	0.52
query10	0.57	0.57	0.56
query11	0.15	0.10	0.10
query12	0.15	0.11	0.11
query13	0.62	0.61	0.62
query14	2.67	2.72	2.73
query15	0.92	0.86	0.85
query16	0.37	0.37	0.38
query17	1.02	1.05	1.01
query18	0.22	0.21	0.20
query19	1.91	1.79	1.97
query20	0.01	0.01	0.01
query21	15.35	0.91	0.54
query22	0.75	1.20	0.75
query23	14.81	1.40	0.63
query24	12.68	0.99	0.42
query25	0.33	0.08	0.08
query26	0.94	0.19	0.14
query27	0.05	0.05	0.04
query28	5.98	0.81	0.45
query29	12.53	3.90	3.26
query30	0.25	0.09	0.06
query31	2.81	0.60	0.39
query32	3.24	0.55	0.47
query33	3.06	3.02	2.99
query34	15.69	5.14	4.57
query35	4.53	4.50	4.53
query36	0.66	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.12
query41	0.06	0.03	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 106.92 s
Total hot run time: 30.25 s

@github-actions
Copy link
Contributor

github-actions bot commented Feb 19, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Feb 19, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Feb 19, 2025

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit 2a76ef9 into apache:master Feb 19, 2025
27 of 28 checks passed
@924060929 924060929 deleted the fix_partition_prune branch February 19, 2025 08:28
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
@924060929 @lzyy2024
[fix](nereids) fix prune partitions wrong (apache#48034)
7f7c126 
fix prune partitions wrong, introduced by apache#44574
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@924060929 @koarz
[fix](nereids) fix prune partitions wrong (apache#48034)
3aac59a 
fix prune partitions wrong, introduced by apache#44574
@englefly englefly mentioned this pull request Jun 18, 2025
Merged
morrySnow pushed a commit that referenced this pull request Jun 20, 2025
@englefly
branch-3.1: [opt](nereids) infer in-predicate from or-predicate (#51869)
622e21e 
pick 
#41222
#46468
#44574
#48034
#50265
#47385
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@924060929 @Thearas @doris-robot @starocean999 @morrySnow