-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Fix](Nereids) Add cast comparison with slot reference when inferring predicate #21171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
run buildall |
c289425 to
eaccc91
Compare
|
run feut |
|
run buildall |
eaccc91 to
81ac254
Compare
|
run buildall |
| // under the License. | ||
|
|
||
| suite("load") { | ||
| sql 'create database if not exists nereids_infer_predicate_test' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not need create database here, remove load.groovy completely
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PredicatePropagation.java
Show resolved
Hide resolved
81ac254 to
d503715
Compare
|
run buildall |
|
TeamCity pipeline, clickbench performance test result: |
d503715 to
b9c4777
Compare
|
run buildall |
|
TeamCity pipeline, clickbench performance test result: |
|
(From new mechine)TeamCity pipeline, clickbench performance test result: |
b9c4777 to
42e2ab1
Compare
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
run buildall |
|
TeamCity pipeline, clickbench performance test result: |
|
run clickbench |
42e2ab1 to
7e28924
Compare
|
run buildall |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
TeamCity pipeline, clickbench performance test result: |
|
TeamCity pipeline, clickbench performance test result: |
9b3a2a8 to
b5c9089
Compare
|
run buildall |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
… predicate (#21171) Problem: When inferring predicate, we assume that slot reference need to be inferred. But in this case: carete table tb1(l1 smallint) ...; create table tb2(l2 int) ...; select * from tb1 inner join tb2 where tb1.l1 = tb2.l2 and tb2.l2 = 1; We can not get tb1.l1 = 1 filter because we will add a cast to l1 (Cast smallint to int l1) = l2. Solved: Add cast consideration when inferring predicate, also add change judgement when judging equals to slotreference and cast expression. But when we want to infer predicate from bigger type cast to smaller type, it is logical error. For example: select * from tb1 inner join tb2 where tb1.l1 = cast(tb2.l2 as smallint) and tb2.l2 = (number between smallint max and intmax); tb2.l2 value can not infer to left side because tb1.l1 would be false value, and when we add one more condition like tb1.l1 = tb3.l3(smallint). It would cause this predicate be false.
… predicate (apache#21171) Problem: When inferring predicate, we assume that slot reference need to be inferred. But in this case: carete table tb1(l1 smallint) ...; create table tb2(l2 int) ...; select * from tb1 inner join tb2 where tb1.l1 = tb2.l2 and tb2.l2 = 1; We can not get tb1.l1 = 1 filter because we will add a cast to l1 (Cast smallint to int l1) = l2. Solved: Add cast consideration when inferring predicate, also add change judgement when judging equals to slotreference and cast expression. But when we want to infer predicate from bigger type cast to smaller type, it is logical error. For example: select * from tb1 inner join tb2 where tb1.l1 = cast(tb2.l2 as smallint) and tb2.l2 = (number between smallint max and intmax); tb2.l2 value can not infer to left side because tb1.l1 would be false value, and when we add one more condition like tb1.l1 = tb3.l3(smallint). It would cause this predicate be false.
This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
…pache#25637) pick from master PR: apache#25637 commit id: ae66464 extract slot and literal in comparison predicate. infer new one by equals predicates. use TypeCoercion to add cast on new comparison predicate to ensure it is correct. This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
…25637) (#25930) pick from master PR: #25637 commit id: ae66464 extract slot and literal in comparison predicate. infer new one by equals predicates. use TypeCoercion to add cast on new comparison predicate to ensure it is correct. This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (#21171)" commit 58f2593.
…pache#25637) extract slot and literal in comparison predicate. infer new one by equals predicates. use TypeCoercion to add cast on new comparison predicate to ensure it is correct. This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
…pache#25637) extract slot and literal in comparison predicate. infer new one by equals predicates. use TypeCoercion to add cast on new comparison predicate to ensure it is correct. This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
…pache#25637) (apache#25930) pick from master PR: apache#25637 commit id: ae66464 extract slot and literal in comparison predicate. infer new one by equals predicates. use TypeCoercion to add cast on new comparison predicate to ensure it is correct. This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
…pache#25637) extract slot and literal in comparison predicate. infer new one by equals predicates. use TypeCoercion to add cast on new comparison predicate to ensure it is correct. This reverts "[Fix](Nereids) Add cast comparison with slot reference when inferring predicate (apache#21171)" commit 58f2593.
Proposed changes
Problem:
When inferring predicate, we assume that slot reference need to be inferred. But in this case:
carete table tb1(l1 smallint) ...;
create table tb2(l2 int) ...;
select * from tb1 inner join tb2 where tb1.l1 = tb2.l2 and tb2.l2 = 1;
We can not get tb1.l1 = 1 filter because we will add a cast to l1 (Cast smallint to int l1) = l2.
Solved:
Add cast consideration when inferring predicate, also add change judgement when judging equals to slotreference and cast expression. But when we want to infer predicate from bigger type cast to smaller type, it is logical error.
For example:
select * from tb1 inner join tb2 where tb1.l1 = cast(tb2.l2 as smallint) and tb2.l2 = (number between smallint max and intmax);
tb2.l2 value can not infer to left side because tb1.l1 would be false value, and when we add one more condition like tb1.l1 = tb3.l3(smallint). It would cause this predicate be false.
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...