Skip to content

branch-3.0: [fix](jdbc catalog) Improve conjunct expression handling in JdbcScanNode #50542 #50647

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
May 14, 2025
Merged

branch-3.0: [fix](jdbc catalog) Improve conjunct expression handling in JdbcScanNode #50542 #50647

dataroaring merged 1 commit into from
May 14, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented May 7, 2025

Cherry-picked from #50542

@zy-kkk
[fix](jdbc catalog) Improve conjunct expression handling in JdbcScanN…
986d450 
…ode (#50542)

Previously, we ignored some BinaryPredicates in CompoundPredicate that may require additional processing, such as `explain select * from TEST_TIMESTAMP where ID = 1 OR (T2 >= CAST(CURDATE() AS DATETIME) AND T1 >= CAST(CURDATE() AS DATETIME));`
@github-actions github-actions bot requested a review from dataroaring as a code owner May 7, 2025 06:40
@Thearas
Copy link
Contributor

Thearas commented May 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this May 7, 2025
@dataroaring dataroaring reopened this May 7, 2025
@Thearas
Copy link
Contributor

Thearas commented May 7, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 7, 2025

TPC-H: Total hot run time: 40647 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 986d45046b2b9cd612247ea364bfb3d16ffa4ed2, data reload: false

------ Round 1 ----------------------------------
q1	17573	7034	6624	6624
q2	2067	171	166	166
q3	10583	1089	1183	1089
q4	10571	765	776	765
q5	7726	2860	2880	2860
q6	218	137	136	136
q7	1004	614	611	611
q8	9356	2011	2036	2011
q9	6630	6402	6454	6402
q10	7045	2257	2319	2257
q11	472	260	269	260
q12	408	222	216	216
q13	17796	3023	2987	2987
q14	245	206	208	206
q15	506	474	460	460
q16	685	606	589	589
q17	998	665	570	570
q18	7453	6809	6811	6809
q19	1442	1106	1060	1060
q20	478	204	215	204
q21	4068	3435	3422	3422
q22	1101	994	943	943
Total cold run time: 108425 ms
Total hot run time: 40647 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6696	6584	6583	6583
q2	326	237	230	230
q3	2933	2821	2960	2821
q4	2014	1816	1764	1764
q5	5784	5744	5773	5744
q6	212	133	129	129
q7	2231	1855	1856	1855
q8	3411	3576	3540	3540
q9	8891	8918	8921	8918
q10	3616	3531	3550	3531
q11	595	501	496	496
q12	832	594	602	594
q13	9525	3185	3183	3183
q14	296	267	273	267
q15	512	466	473	466
q16	717	648	677	648
q17	1877	1640	1609	1609
q18	8317	7807	7648	7648
q19	1686	1548	1592	1548
q20	2111	1908	1836	1836
q21	5615	5338	5468	5338
q22	1163	1032	994	994
Total cold run time: 69360 ms
Total hot run time: 59742 ms

@doris-robot
Copy link

doris-robot commented May 7, 2025

TPC-DS: Total hot run time: 197402 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 986d45046b2b9cd612247ea364bfb3d16ffa4ed2, data reload: false

query1	1302	906	885	885
query2	6221	2069	1984	1984
query3	10784	4303	4347	4303
query4	65949	27969	23418	23418
query5	5071	458	456	456
query6	427	195	176	176
query7	5680	320	315	315
query8	305	234	220	220
query9	9299	2664	2630	2630
query10	484	277	267	267
query11	18177	15203	15913	15203
query12	161	108	109	108
query13	1559	458	445	445
query14	10222	7377	7221	7221
query15	203	180	182	180
query16	7135	509	460	460
query17	1112	603	577	577
query18	1842	341	324	324
query19	237	163	166	163
query20	121	107	106	106
query21	217	111	103	103
query22	4691	4402	4664	4402
query23	34810	34009	35005	34009
query24	6319	2968	2954	2954
query25	532	421	452	421
query26	655	180	173	173
query27	1729	355	357	355
query28	4239	2496	2450	2450
query29	727	463	461	461
query30	247	159	162	159
query31	1011	810	818	810
query32	67	56	54	54
query33	453	293	279	279
query34	913	511	526	511
query35	844	747	710	710
query36	1079	948	979	948
query37	122	65	66	65
query38	4126	3937	3985	3937
query39	1515	1517	1480	1480
query40	207	104	101	101
query41	49	49	46	46
query42	122	101	103	101
query43	536	495	496	495
query44	1244	811	822	811
query45	188	163	171	163
query46	1143	743	752	743
query47	2064	1927	1970	1927
query48	473	389	392	389
query49	734	403	393	393
query50	869	417	432	417
query51	7280	7306	7285	7285
query52	111	89	93	89
query53	281	182	196	182
query54	592	489	469	469
query55	81	78	74	74
query56	271	247	256	247
query57	1259	1161	1133	1133
query58	212	209	214	209
query59	3160	2924	2874	2874
query60	273	248	260	248
query61	112	144	136	136
query62	794	669	671	669
query63	217	190	190	190
query64	1389	697	650	650
query65	3252	3239	3195	3195
query66	708	291	294	291
query67	15913	15661	15662	15661
query68	4084	596	568	568
query69	432	262	262	262
query70	1192	1136	1109	1109
query71	367	261	254	254
query72	6102	4018	4020	4018
query73	770	346	355	346
query74	10597	8971	9315	8971
query75	3382	2636	2644	2636
query76	1894	1127	1056	1056
query77	491	280	276	276
query78	10585	9665	9547	9547
query79	1617	595	596	595
query80	874	429	434	429
query81	522	238	239	238
query82	1270	94	91	91
query83	194	146	149	146
query84	274	90	84	84
query85	875	312	292	292
query86	351	304	300	300
query87	4402	4285	4232	4232
query88	3866	2390	2364	2364
query89	411	294	301	294
query90	2001	192	188	188
query91	180	148	151	148
query92	63	51	51	51
query93	1973	567	551	551
query94	802	296	301	296
query95	358	253	257	253
query96	614	284	283	283
query97	3288	3126	3177	3126
query98	210	203	194	194
query99	1559	1296	1294	1294
Total cold run time: 319721 ms
Total hot run time: 197402 ms

@doris-robot
Copy link

doris-robot commented May 7, 2025

ClickBench: Total hot run time: 32.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 986d45046b2b9cd612247ea364bfb3d16ffa4ed2, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.63	0.10	0.10
query5	0.55	0.52	0.54
query6	1.12	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.03
query9	0.56	0.51	0.49
query10	0.56	0.56	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.59	0.59
query14	2.76	2.72	2.73
query15	0.89	0.84	0.83
query16	0.38	0.39	0.39
query17	1.02	1.04	1.00
query18	0.23	0.22	0.21
query19	1.92	1.81	2.08
query20	0.01	0.01	0.01
query21	15.36	0.61	0.59
query22	2.49	1.96	1.70
query23	16.85	1.06	0.79
query24	3.12	1.26	1.95
query25	0.32	0.16	0.07
query26	0.43	0.15	0.13
query27	0.04	0.05	0.04
query28	9.26	0.55	0.50
query29	12.61	3.31	3.25
query30	0.25	0.06	0.06
query31	2.85	0.40	0.38
query32	3.24	0.46	0.46
query33	2.98	3.01	3.06
query34	17.06	4.47	4.48
query35	4.56	4.49	4.48
query36	0.66	0.50	0.50
query37	0.08	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.02	0.03
query40	0.17	0.12	0.13
query41	0.08	0.02	0.03
query42	0.04	0.02	0.03
query43	0.03	0.03	0.04
Total cold run time: 105.5 s
Total hot run time: 32.3 s

dataroaring
dataroaring approved these changes May 14, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 9d4ecf7 into branch-3.0 May 14, 2025
23 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-50542-branch-3.0 branch May 14, 2025 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @doris-robot @dataroaring @zy-kkk