Skip to content

[fix](Nereids): support evaluate join that has one side condition #33702

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Apr 17, 2024
Merged

[fix](Nereids): support evaluate join that has one side condition #33702

morrySnow merged 1 commit into from
Apr 17, 2024

Conversation

@keanji-x
Copy link
Contributor

@keanji-x keanji-x commented Apr 16, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Apr 16, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@keanji-x
Copy link
Contributor Author

keanji-x commented Apr 16, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Apr 16, 2024

TPC-H: Total hot run time: 38481 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 485c95d54c16b018d3f29edd7c6e0b586cfd7ba2, data reload: false

------ Round 1 ----------------------------------
q1	17819	4435	4376	4376
q2	2510	194	194	194
q3	10875	1332	1218	1218
q4	10197	695	829	695
q5	7509	2669	2675	2669
q6	218	133	130	130
q7	1003	580	577	577
q8	9221	2050	2037	2037
q9	7369	6617	6521	6521
q10	8471	3540	3499	3499
q11	452	230	230	230
q12	392	224	222	222
q13	17767	2955	2908	2908
q14	284	232	224	224
q15	520	472	481	472
q16	533	405	381	381
q17	971	571	598	571
q18	7344	6806	6797	6797
q19	1607	1532	1513	1513
q20	641	312	305	305
q21	3479	2840	2639	2639
q22	363	316	303	303
Total cold run time: 109545 ms
Total hot run time: 38481 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4235	4191	4221	4191
q2	381	267	276	267
q3	2975	2727	2755	2727
q4	1884	1607	1535	1535
q5	5339	5305	5285	5285
q6	208	126	122	122
q7	2269	1871	1876	1871
q8	3228	3369	3373	3369
q9	8537	8585	8589	8585
q10	3900	3723	3653	3653
q11	583	472	477	472
q12	770	609	569	569
q13	17217	2919	2891	2891
q14	297	268	271	268
q15	509	470	475	470
q16	473	414	433	414
q17	1776	1452	1448	1448
q18	7621	7437	7457	7437
q19	1637	1561	1524	1524
q20	1945	1736	1725	1725
q21	4943	4680	4793	4680
q22	522	472	456	456
Total cold run time: 71249 ms
Total hot run time: 53959 ms

@doris-robot
Copy link

doris-robot commented Apr 16, 2024

TPC-DS: Total hot run time: 182773 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 485c95d54c16b018d3f29edd7c6e0b586cfd7ba2, data reload: false

query1	905	374	362	362
query2	6511	2515	2409	2409
query3	6651	209	213	209
query4	23425	21311	21411	21311
query5	4134	415	415	415
query6	269	175	177	175
query7	4595	286	284	284
query8	223	174	168	168
query9	8432	2325	2323	2323
query10	580	264	253	253
query11	14817	14167	14242	14167
query12	137	90	85	85
query13	1648	376	367	367
query14	10000	7902	7815	7815
query15	254	184	182	182
query16	8147	263	260	260
query17	1929	594	572	572
query18	2124	274	275	274
query19	301	148	153	148
query20	93	90	85	85
query21	205	122	120	120
query22	4970	4888	4802	4802
query23	33731	33069	33267	33069
query24	10832	2894	2902	2894
query25	617	363	393	363
query26	1499	151	149	149
query27	2912	296	302	296
query28	7609	2008	1991	1991
query29	915	594	594	594
query30	268	171	168	168
query31	984	716	736	716
query32	92	54	52	52
query33	745	247	235	235
query34	1045	462	468	462
query35	840	666	681	666
query36	1028	913	928	913
query37	136	69	71	69
query38	3363	3196	3196	3196
query39	1578	1551	1526	1526
query40	279	120	122	120
query41	46	44	46	44
query42	105	95	100	95
query43	602	559	560	559
query44	1206	717	714	714
query45	258	265	263	263
query46	1081	751	754	751
query47	1944	1862	1860	1860
query48	349	295	304	295
query49	1125	368	378	368
query50	731	374	377	374
query51	6754	6533	6618	6533
query52	102	86	95	86
query53	345	284	278	278
query54	307	232	225	225
query55	78	71	70	70
query56	240	225	219	219
query57	1212	1134	1129	1129
query58	218	196	194	194
query59	3414	3203	3233	3203
query60	253	234	257	234
query61	88	87	85	85
query62	643	446	441	441
query63	305	277	279	277
query64	6392	3871	3061	3061
query65	3201	3041	3054	3041
query66	1400	331	342	331
query67	15432	14951	14857	14857
query68	5237	528	521	521
query69	482	296	297	296
query70	1182	1165	1107	1107
query71	387	275	258	258
query72	6332	2599	2429	2429
query73	701	317	310	310
query74	6889	6384	6329	6329
query75	3328	2691	2598	2598
query76	2833	962	1012	962
query77	382	269	263	263
query78	10893	10340	10198	10198
query79	3346	523	539	523
query80	2051	428	425	425
query81	536	250	241	241
query82	781	96	102	96
query83	270	171	169	169
query84	267	87	88	87
query85	2111	274	284	274
query86	487	280	288	280
query87	3530	3292	3304	3292
query88	4699	2342	2344	2342
query89	490	375	368	368
query90	2012	189	247	189
query91	126	97	94	94
query92	63	48	47	47
query93	4911	517	496	496
query94	1238	173	176	173
query95	381	300	297	297
query96	582	265	266	265
query97	3194	2935	2962	2935
query98	235	260	217	217
query99	1221	869	840	840
Total cold run time: 287831 ms
Total hot run time: 182773 ms

@doris-robot
Copy link

doris-robot commented Apr 16, 2024

ClickBench: Total hot run time: 30.59 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 485c95d54c16b018d3f29edd7c6e0b586cfd7ba2, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.08
query5	0.49	0.52	0.50
query6	1.46	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.49
query10	0.55	0.55	0.55
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.76	0.78	0.78
query15	0.82	0.81	0.81
query16	0.36	0.36	0.36
query17	1.03	1.00	1.01
query18	0.21	0.26	0.23
query19	1.88	1.72	1.80
query20	0.02	0.01	0.00
query21	15.42	0.65	0.64
query22	4.88	6.19	2.14
query23	18.31	1.41	1.33
query24	1.31	0.44	0.22
query25	0.15	0.08	0.07
query26	0.28	0.16	0.16
query27	0.08	0.07	0.08
query28	13.28	1.00	0.98
query29	12.62	3.26	3.28
query30	0.26	0.06	0.05
query31	2.88	0.37	0.37
query32	3.29	0.46	0.46
query33	2.82	2.82	2.81
query34	16.99	4.34	4.40
query35	4.47	4.42	4.52
query36	0.65	0.47	0.46
query37	0.19	0.15	0.15
query38	0.15	0.14	0.15
query39	0.05	0.04	0.04
query40	0.18	0.14	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.56 s
Total hot run time: 30.59 s

@doris-robot
Copy link

doris-robot commented Apr 16, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 485c95d54c16b018d3f29edd7c6e0b586cfd7ba2 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      34 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

@github-actions
Copy link
Contributor

github-actions bot commented Apr 17, 2024

PR approved by anyone and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 17, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Apr 17, 2024

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit 7797ec9 into apache:master Apr 17, 2024
yiguolei pushed a commit that referenced this pull request Apr 17, 2024
@keanji-x @Doris-Extras
[fix](Nereids) dphyper support evaluate join that has one side condit…
5985266 
…ion (#33702)
yiguolei pushed a commit that referenced this pull request Apr 17, 2024
@keanji-x @Doris-Extras
[fix](Nereids) dphyper support evaluate join that has one side condit…
048448e 
…ion (#33702)
dataroaring pushed a commit that referenced this pull request Apr 24, 2024
@keanji-x @dataroaring
[fix](Nereids) dphyper support evaluate join that has one side condit…
a054804 
…ion (#33702)
keanji-x added a commit to keanji-x/doris that referenced this pull request May 23, 2024
@keanji-x
[fix](Nereids) dphyper support evaluate join that has one side condit…
be40fa8 
…ion (apache#33702)
@keanji-x keanji-x mentioned this pull request May 23, 2024
Merged
keanji-x added a commit to keanji-x/doris that referenced this pull request May 23, 2024
@keanji-x
[fix](Nereids) dphyper support evaluate join that has one side condit…
c501751 
…ion (apache#33702)
keanji-x added a commit to keanji-x/doris that referenced this pull request May 24, 2024
@keanji-x
[fix](Nereids) dphyper support evaluate join that has one side condit…
69a0f10 
…ion (apache#33702)
keanji-x added a commit to keanji-x/doris that referenced this pull request May 24, 2024
@keanji-x
[fix](Nereids) dphyper support evaluate join that has one side condit…
2f1fdb5 
…ion (apache#33702)
xiaokang pushed a commit that referenced this pull request May 26, 2024
@keanji-x
[fix](Nereids) dphyper support evaluate join that has one side condit…
0496785 
…ion (#33702) (#35268)
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@keanji-x
[fix](Nereids) dphyper support evaluate join that has one side condit…
7741bf5 
…ion (apache#33702) (apache#35268)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

+1 more reviewer

@LiBinfeng-01 LiBinfeng-01 LiBinfeng-01 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.11-merged dev/2.1.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@keanji-x @doris-robot @LiBinfeng-01 @morrySnow @xiaokang