Skip to content

Conversation

@feiniaofeiafei
Copy link
Contributor

@feiniaofeiafei feiniaofeiafei commented May 6, 2024

The legacy planner encounters issues when handling filters such as: c1(boolean type)=0.0(decimalv3).
The literal 0.0 is interpreted as decimalv3(1,1), and the boolean type c1 is coerced to decimalv3(1,1).
decimalv3(1,1) can only retain values in the range [0,1), while the boolean true is represented as 1, exceeding the upper bound, thus causing an overflow problem.
This pull request addresses this issue by considering the boolean type as decimalv3(1,0), making both c1 and 0.0 being cast to decimal(2,1).

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40119 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bfeef530d8b355debca4d63416be6d9b54e6e129, data reload: false

------ Round 1 ----------------------------------
q1	17652	4364	4246	4246
q2	2024	191	191	191
q3	10673	1317	1223	1223
q4	10384	872	828	828
q5	7660	2779	2582	2582
q6	214	133	135	133
q7	1061	608	592	592
q8	9582	2167	2146	2146
q9	9183	6802	6669	6669
q10	9293	3713	3672	3672
q11	448	242	225	225
q12	458	211	210	210
q13	17758	2939	2984	2939
q14	273	224	239	224
q15	515	486	470	470
q16	506	383	375	375
q17	978	653	677	653
q18	8066	7354	7501	7354
q19	2661	1543	1491	1491
q20	648	301	305	301
q21	5201	3845	3324	3324
q22	342	271	276	271
Total cold run time: 115580 ms
Total hot run time: 40119 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4398	4219	4243	4219
q2	377	258	261	258
q3	2979	2704	2749	2704
q4	1858	1557	1589	1557
q5	5291	5247	5294	5247
q6	215	125	122	122
q7	2267	1901	1927	1901
q8	3207	3399	3381	3381
q9	8500	8496	8486	8486
q10	3890	3711	3664	3664
q11	583	482	478	478
q12	751	577	601	577
q13	17289	2985	2975	2975
q14	301	281	265	265
q15	531	479	473	473
q16	464	412	434	412
q17	1754	1471	1440	1440
q18	7583	7502	7561	7502
q19	1696	1537	1580	1537
q20	1953	1757	1755	1755
q21	4966	4941	4944	4941
q22	569	491	506	491
Total cold run time: 71422 ms
Total hot run time: 54385 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185740 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bfeef530d8b355debca4d63416be6d9b54e6e129, data reload: false

query1	910	359	340	340
query2	6798	2662	2417	2417
query3	6657	208	208	208
query4	23014	21208	21261	21208
query5	4231	415	408	408
query6	262	177	183	177
query7	4600	281	278	278
query8	242	195	197	195
query9	8651	2351	2305	2305
query10	420	242	251	242
query11	14785	14125	14229	14125
query12	135	93	89	89
query13	1637	363	368	363
query14	9727	7304	7591	7304
query15	227	164	173	164
query16	7936	258	261	258
query17	1845	550	549	549
query18	2034	276	259	259
query19	203	153	151	151
query20	90	88	90	88
query21	194	127	125	125
query22	5091	4867	4936	4867
query23	33785	33311	33185	33185
query24	11873	2861	2870	2861
query25	648	358	359	358
query26	1792	159	149	149
query27	3027	321	323	321
query28	7704	2018	2005	2005
query29	1059	618	596	596
query30	298	149	151	149
query31	983	760	740	740
query32	91	51	54	51
query33	736	243	242	242
query34	1047	479	468	468
query35	812	683	664	664
query36	1058	946	944	944
query37	175	68	69	68
query38	3187	3078	2996	2996
query39	1586	1551	1553	1551
query40	285	127	129	127
query41	44	41	41	41
query42	106	95	99	95
query43	598	557	539	539
query44	1217	726	733	726
query45	272	245	247	245
query46	1075	715	744	715
query47	1982	1855	1863	1855
query48	383	305	301	301
query49	1219	406	404	404
query50	772	380	380	380
query51	6797	6636	6610	6610
query52	103	129	96	96
query53	347	278	277	277
query54	310	232	237	232
query55	78	71	71	71
query56	242	221	225	221
query57	1228	1123	1165	1123
query58	253	196	197	196
query59	3464	3188	3207	3188
query60	256	231	231	231
query61	91	86	87	86
query62	667	458	455	455
query63	313	273	275	273
query64	9606	7219	7210	7210
query65	3083	3079	3066	3066
query66	1341	339	331	331
query67	15265	15324	14898	14898
query68	5252	538	527	527
query69	496	305	328	305
query70	1144	1139	1099	1099
query71	418	272	257	257
query72	7194	2594	2346	2346
query73	708	314	316	314
query74	6494	6024	5978	5978
query75	3407	2635	2632	2632
query76	3446	969	982	969
query77	391	260	264	260
query78	11023	10173	10233	10173
query79	6144	511	538	511
query80	1721	427	438	427
query81	537	235	218	218
query82	885	93	99	93
query83	314	173	166	166
query84	270	84	83	83
query85	2019	271	307	271
query86	487	318	318	318
query87	3264	3058	3094	3058
query88	5002	2341	2321	2321
query89	498	384	378	378
query90	2012	185	184	184
query91	127	97	101	97
query92	66	49	48	48
query93	5192	519	499	499
query94	1241	184	189	184
query95	393	314	302	302
query96	587	261	259	259
query97	3147	2923	2991	2923
query98	244	229	217	217
query99	1203	894	863	863
Total cold run time: 296273 ms
Total hot run time: 185740 ms

morrySnow
morrySnow previously approved these changes May 6, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 6, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented May 6, 2024

PR approved by anyone and no changes requested.

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label May 8, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 8, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 8, 2024
@feiniaofeiafei feiniaofeiafei requested a review from morrySnow May 8, 2024 13:42
@morrySnow morrySnow changed the title [Fix](nereids) fix ScalarType.getAssignmentCompatibleType() when deal boolean and decimal [Fix](planner) fix ScalarType.getAssignmentCompatibleType() when deal boolean and decimal May 9, 2024
@morrySnow morrySnow merged commit c472206 into apache:master May 9, 2024
yiguolei pushed a commit that referenced this pull request May 10, 2024
… boolean and decimal (#34435)

The legacy planner encounters issues when handling filters such as: c1(boolean type)=0.0(decimalv3).
The literal 0.0 is interpreted as decimalv3(1,1), and the boolean type c1 is coerced to decimalv3(1,1).
decimalv3(1,1) can only retain values in the range [0,1), while the boolean true is represented as 1, exceeding the upper bound, thus causing an overflow problem.
This pull request addresses this issue by considering the boolean type as decimalv3(1,0), making both c1 and 0.0 being cast to decimal(2,1).


Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
ByteYue pushed a commit to ByteYue/doris that referenced this pull request May 15, 2024
… boolean and decimal (apache#34435)

The legacy planner encounters issues when handling filters such as: c1(boolean type)=0.0(decimalv3).
The literal 0.0 is interpreted as decimalv3(1,1), and the boolean type c1 is coerced to decimalv3(1,1).
decimalv3(1,1) can only retain values in the range [0,1), while the boolean true is represented as 1, exceeding the upper bound, thus causing an overflow problem.
This pull request addresses this issue by considering the boolean type as decimalv3(1,0), making both c1 and 0.0 being cast to decimal(2,1).


Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
morrySnow pushed a commit that referenced this pull request May 15, 2024
… boolean and decimal (#34636)

cherry-pick to branch-2.0 from #34435

The legacy planner encounters issues when handling filters such as: c1(boolean type)=0.0(decimalv3).
The literal 0.0 is interpreted as decimalv3(1,1), and the boolean type c1 is coerced to decimalv3(1,1).
decimalv3(1,1) can only retain values in the range [0,1), while the boolean true is represented as 1, exceeding the upper bound, thus causing an overflow problem.
This pull request addresses this issue by considering the boolean type as decimalv3(1,0), making both c1 and 0.0 being cast to decimal(2,1).
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
… boolean and decimal (apache#34636)

cherry-pick to branch-2.0 from apache#34435

The legacy planner encounters issues when handling filters such as: c1(boolean type)=0.0(decimalv3).
The literal 0.0 is interpreted as decimalv3(1,1), and the boolean type c1 is coerced to decimalv3(1,1).
decimalv3(1,1) can only retain values in the range [0,1), while the boolean true is represented as 1, exceeding the upper bound, thus causing an overflow problem.
This pull request addresses this issue by considering the boolean type as decimalv3(1,0), making both c1 and 0.0 being cast to decimal(2,1).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants