Skip to content

Conversation

@seawinde
Copy link
Contributor

Proposed changes

In #36111, we add isDeterministic method in class ExpressionTrait to identify the expression is deterministic or not.

But unix_timestamp doesn't extend Nondeterministic, but it is not deterministic when it's children is empty. and is not deterministic when children is not empty. If we useinstanceOf Nondeterministicto indentify if expression is is not deterministic, that is confused.

So we do something as fllowing:

  1. Remove Nondeterministic class, and use isDeterministic to indentify it's deterministic.
  2. Add containsNondeterministic method in ExpressionTrait to identify it contains nondeterministic expression or not.
  3. isDeterministic only identify current expression is deterministic or not. would identify if contains nondeterministic or not

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Comment on lines 399 to 400
|| cachedVariable.getRealExpression().anyMatch(expr -> expr instanceof ExpressionTrait
&& !((ExpressionTrait) expr).isDeterministic())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove expr instanceof ExpressionTrait because Expression is always ExpressionTrait

* Identify the expression is containing deterministic expr or not
*/
default boolean containsNondeterministic() {
return anyMatch(expr -> expr instanceof ExpressionTrait && !((ExpressionTrait) expr).isDeterministic());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove expr instanceof ExpressionTrait

@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37853 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 25170801ca5a1e33f5f3c3d5c04d2aad09037204, data reload: false

------ Round 1 ----------------------------------
q1	17606	4375	4305	4305
q2	2008	179	176	176
q3	11707	970	1164	970
q4	10515	755	693	693
q5	7753	2865	2812	2812
q6	229	139	140	139
q7	984	608	591	591
q8	9321	2078	2078	2078
q9	7276	6515	6534	6515
q10	6992	2180	2210	2180
q11	450	242	243	242
q12	387	219	226	219
q13	18945	3040	3043	3040
q14	285	232	237	232
q15	523	480	479	479
q16	487	400	388	388
q17	998	679	763	679
q18	7217	6911	6802	6802
q19	1405	998	1063	998
q20	700	320	330	320
q21	3954	3043	2996	2996
q22	1108	1027	999	999
Total cold run time: 110850 ms
Total hot run time: 37853 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4341	4317	4291	4291
q2	383	271	265	265
q3	2891	2660	2681	2660
q4	1955	1620	1637	1620
q5	5606	5661	5736	5661
q6	220	142	145	142
q7	2184	1825	1800	1800
q8	3337	3445	3458	3445
q9	8805	8816	8817	8816
q10	3589	3384	3329	3329
q11	624	532	508	508
q12	848	679	679	679
q13	15645	3159	3242	3159
q14	322	303	282	282
q15	544	485	486	485
q16	480	451	443	443
q17	1829	1543	1540	1540
q18	8170	7868	7803	7803
q19	1743	1463	1605	1463
q20	2146	1942	1923	1923
q21	5774	5369	5472	5369
q22	1142	1044	1008	1008
Total cold run time: 72578 ms
Total hot run time: 56691 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191496 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 25170801ca5a1e33f5f3c3d5c04d2aad09037204, data reload: false

query1	1251	904	872	872
query2	6285	1905	1884	1884
query3	10634	3880	3963	3880
query4	59953	26051	23449	23449
query5	5352	502	499	499
query6	410	167	163	163
query7	5909	292	287	287
query8	285	208	220	208
query9	8995	2486	2479	2479
query10	498	278	257	257
query11	17918	15094	15358	15094
query12	158	100	100	100
query13	1546	395	382	382
query14	11058	7454	7121	7121
query15	230	170	174	170
query16	7523	471	476	471
query17	1117	594	570	570
query18	1946	300	292	292
query19	275	145	145	145
query20	128	113	116	113
query21	206	100	105	100
query22	4686	4418	4409	4409
query23	34646	33567	33366	33366
query24	5847	2866	2792	2792
query25	491	375	374	374
query26	679	155	154	154
query27	1784	299	273	273
query28	3737	2047	2028	2028
query29	637	407	405	405
query30	237	146	149	146
query31	930	768	755	755
query32	79	54	56	54
query33	448	287	285	285
query34	860	460	477	460
query35	828	733	704	704
query36	1048	934	932	932
query37	141	86	80	80
query38	3933	3933	3809	3809
query39	1450	1381	1409	1381
query40	193	115	113	113
query41	48	43	43	43
query42	112	93	94	93
query43	520	470	464	464
query44	1078	730	739	730
query45	197	162	164	162
query46	1096	743	722	722
query47	1875	1797	1833	1797
query48	364	287	287	287
query49	776	421	416	416
query50	807	414	407	407
query51	7072	6976	6959	6959
query52	97	87	90	87
query53	266	186	179	179
query54	559	442	441	441
query55	77	73	74	73
query56	271	250	258	250
query57	1136	1090	1081	1081
query58	225	242	236	236
query59	3077	2832	2862	2832
query60	288	266	263	263
query61	99	94	97	94
query62	758	659	669	659
query63	211	180	183	180
query64	4258	2250	1733	1733
query65	3230	3203	3139	3139
query66	603	343	338	338
query67	15438	15276	15312	15276
query68	2973	556	562	556
query69	388	274	274	274
query70	1166	1131	1067	1067
query71	363	275	271	271
query72	2611	2084	2050	2050
query73	696	314	317	314
query74	9145	8919	8815	8815
query75	3361	2727	2659	2659
query76	1425	984	1046	984
query77	560	329	323	323
query78	9591	9218	9006	9006
query79	1050	529	517	517
query80	674	488	488	488
query81	466	234	228	228
query82	292	130	129	129
query83	169	147	150	147
query84	263	73	74	73
query85	665	290	281	281
query86	305	261	305	261
query87	4303	4367	4261	4261
query88	3000	2299	2300	2299
query89	379	286	284	284
query90	1925	201	200	200
query91	126	169	102	102
query92	58	52	51	51
query93	1007	526	538	526
query94	710	286	302	286
query95	324	262	264	262
query96	586	268	276	268
query97	3179	3090	3054	3054
query98	216	213	206	206
query99	1491	1257	1292	1257
Total cold run time: 302336 ms
Total hot run time: 191496 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.42 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 25170801ca5a1e33f5f3c3d5c04d2aad09037204, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.23	0.05	0.05
query4	1.67	0.10	0.08
query5	0.50	0.49	0.48
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.48	0.48
query10	0.53	0.54	0.53
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.58	0.60
query14	0.76	0.79	0.81
query15	0.84	0.83	0.82
query16	0.37	0.36	0.36
query17	0.94	1.06	1.06
query18	0.21	0.20	0.21
query19	1.83	1.73	1.83
query20	0.00	0.01	0.01
query21	15.39	0.65	0.65
query22	4.12	7.97	1.56
query23	18.27	1.45	1.29
query24	2.14	0.22	0.22
query25	0.15	0.08	0.07
query26	0.27	0.19	0.18
query27	0.07	0.08	0.07
query28	13.23	1.01	1.01
query29	12.62	3.38	3.34
query30	0.24	0.06	0.06
query31	2.87	0.41	0.40
query32	3.26	0.48	0.47
query33	2.94	2.98	3.04
query34	16.97	4.35	4.37
query35	4.42	4.46	4.47
query36	0.65	0.46	0.48
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.03	0.03
query40	0.16	0.12	0.13
query41	0.10	0.05	0.04
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.03 s
Total hot run time: 30.42 s

@seawinde seawinde requested a review from 924060929 August 23, 2024 06:46
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 23, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@starocean999 starocean999 merged commit 2b715a7 into apache:master Aug 26, 2024
BiteTheDDDDt pushed a commit that referenced this pull request Aug 27, 2024
…m_bytes foldable properties (#39944)

## Proposed changes

fix the conflict between #39801 and #39891
seawinde added a commit to seawinde/doris that referenced this pull request Aug 29, 2024
…a expression is nondeterministic or not (apache#39801)

## Proposed changes

In apache#36111, we add `isDeterministic`
method in class `ExpressionTrait` to identify the expression is
deterministic or not.

But `unix_timestamp` doesn't extend Nondeterministic, but it is not
deterministic when it's children is empty. and is not deterministic when
children is not empty. If we use` instanceOf Nondeterministic `to
indentify if expression is is not deterministic, that is confused.

So we do something as fllowing:
1. Remove Nondeterministic class, and use `isDeterministic` to indentify
it's deterministic.
2. Add `containsNondeterministic` method in `ExpressionTrait` to
identify it contains nondeterministic expression or not.
3. `isDeterministic` only identify current expression is deterministic
or not. would identify if contains nondeterministic or not
yiguolei pushed a commit to seawinde/doris that referenced this pull request Aug 29, 2024
…a expression is nondeterministic or not (apache#39801)

## Proposed changes

In apache#36111, we add `isDeterministic`
method in class `ExpressionTrait` to identify the expression is
deterministic or not.

But `unix_timestamp` doesn't extend Nondeterministic, but it is not
deterministic when it's children is empty. and is not deterministic when
children is not empty. If we use` instanceOf Nondeterministic `to
indentify if expression is is not deterministic, that is confused.

So we do something as fllowing:
1. Remove Nondeterministic class, and use `isDeterministic` to indentify
it's deterministic.
2. Add `containsNondeterministic` method in `ExpressionTrait` to
identify it contains nondeterministic expression or not.
3. `isDeterministic` only identify current expression is deterministic
or not. would identify if contains nondeterministic or not
yiguolei pushed a commit that referenced this pull request Aug 29, 2024
… a expression is nondeterministic or not (#39801) (#40130)

## Proposed changes

commitId: 2b715a7
pr: #39801
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Sep 11, 2024
…m_bytes foldable properties (apache#39944)

## Proposed changes

fix the conflict between apache#39801 and apache#39891
dataroaring pushed a commit that referenced this pull request Sep 26, 2024
…a expression is nondeterministic or not (#39801)

## Proposed changes

In #36111, we add `isDeterministic`
method in class `ExpressionTrait` to identify the expression is
deterministic or not.

But `unix_timestamp` doesn't extend Nondeterministic, but it is not
deterministic when it's children is empty. and is not deterministic when
children is not empty. If we use` instanceOf Nondeterministic `to
indentify if expression is is not deterministic, that is confused.

So we do something as fllowing:
1. Remove Nondeterministic class, and use `isDeterministic` to indentify
it's deterministic.
2. Add `containsNondeterministic` method in `ExpressionTrait` to
identify it contains nondeterministic expression or not.
3. `isDeterministic` only identify current expression is deterministic
or not. would identify if contains nondeterministic or not
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
…m_bytes foldable properties (#39944)

## Proposed changes

fix the conflict between #39801 and #39891
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged p0_b reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants