-
Notifications
You must be signed in to change notification settings - Fork 1.9k
fix NULL <op> column evaluation, tests for same
#2510
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| // when a null array is generated for a statistics column, | ||
| assert_eq!(row_group_filter, vec![true, true]); | ||
|
|
||
| // bool = NULL always evaluates to NULL (and thus will not |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is the only surprising change I thought
| "+---------------------+", | ||
| // we expect all the following queries to yield a two null values | ||
| let cases = vec![ | ||
| // 1. Numeric comparison with NULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted to test a bunch of different combinations, so I rewrote the test so I could check them all
| // ---- a different evaluation path) | ||
| // ---- | ||
|
|
||
| // 1. Numeric comparison with NULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests below here are new. The tests above were pre-existing
| array: &dyn Array, | ||
| scalar: &ScalarValue, | ||
| ) -> Result<Option<Result<ArrayRef>>> { | ||
| let bool_type = &DataType::Boolean; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changes here were to make the code more readable (so cargo fmt put it on the same line)
| Operator::GtEq => binary_array_op_scalar!(array, scalar.clone(), lt_eq), | ||
| Operator::Eq => binary_array_op_scalar!(array, scalar.clone(), eq), | ||
| Operator::Lt => { | ||
| binary_array_op_dyn_scalar!(array, scalar.clone(), gt, bool_type) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the actual fix -- to use binary_array_op_dyn_scalar which @WinkerDu updated to handle null constants rather than binary_array_op_scalar which does not.
I plan to look into removing binary_array_op_scalar completely as a follow on PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for late response, love this fix ❤️
|
Sorry for the late review, LGTM! |
|
Thanks @yjshen |
This reverts commit d497cde
Which issue does this PR close?
re #1179 and #2482 .
Rationale for this change
While adding tests suggested in #2481 for reversed arguments, it uncovered a bug if the argument order was different.
Previously we tested
column1 < NULLbut notNULL < column1. When I did so I gotWhat changes are included in this PR?
This PR adds a test exercising the order of expressions with null constants, and fixes evaluation of same
Are there any user-facing changes?
Fewer internal errors
cc @WinkerDu and @yjshen