Skip to content

Fix: support NULL input for regular expression comparison operations#11985

Merged
alamb merged 1 commit intoapache:mainfrom
HuSen8891:datafusion
Aug 15, 2024
Merged

Fix: support NULL input for regular expression comparison operations#11985
alamb merged 1 commit intoapache:mainfrom
HuSen8891:datafusion

Conversation

@HuSen8891
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Related with #11623

Rationale for this change

Regular expression comparison operations accept NULL input, like

select 'abc' ~ null;
or
select null ~ null;

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions Bot added the physical-expr Changes to the physical-expr crates label Aug 14, 2024
Copy link
Copy Markdown
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @HuSen8891 for the contribution 🙏

Can you please add some end to end tests, in sqllogicteset format?

Here are the instructions: https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest

Ideally you should be able to extend one of the existing test files in https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest/test_files

Perhaps https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/test_files/regexp.slt

Comment on lines +1055 to +1057
(DataType::Null, Utf8View | Utf8 | LargeUtf8) => Some(rhs_type.clone()),
(Utf8View | Utf8 | LargeUtf8, DataType::Null) => Some(lhs_type.clone()),
(DataType::Null, DataType::Null) => Some(Utf8),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is any reason to explicitly list out the types?

Like maybe you could simplify this a bit like

let lhs_null = matches!(lhs_type, DataType::Null);
let rhs_null = matches!(rhs_type, DataType::Null);
math (lhs_null, rhs_null) {
  (true, false) => Some(rhs_type.clone()),
  (false, true) => Some(lhs_type.clone())
  (true, true) => Some(Utf8)
  (false, false) => None,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we only accept string and NULL input, not other type, such as select 'abc' ~ null;
But select 1 ~ null is not supported.

@github-actions github-actions Bot added the sqllogictest SQL Logic Tests (.slt) label Aug 15, 2024
@HuSen8891
Copy link
Copy Markdown
Contributor Author

Copy link
Copy Markdown
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @HuSen8891

@alamb alamb merged commit 19ad53d into apache:main Aug 15, 2024
@alamb
Copy link
Copy Markdown
Contributor

alamb commented Aug 15, 2024

Thanks again @HuSen8891

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants