udf: replace function #4347

xinghuayu007 · 2020-08-13T14:13:40Z

Proposed changes

replace is an user defined function, which is to replace all old substrings with a new substring in a string, as follow:
mysql> select replace("http://www.baidu.com:9090", "9090", "");
+------------------------------------------------------+
| replace('http://www.baidu.com:9090', '9090', '') |
+------------------------------------------------------+
| http://www.baidu.com: |
+------------------------------------------------------+

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[] Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[] Documentation Update (if none of the other choices apply)
[] Code refactor (Modify the code structure, format the code, etc...)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[] I have create an issue on (Fix #ISSUE), and have described the bug/feature there in detail
Compiling and unit tests pass locally with my changes
I have added tests that prove my fix is effective or that my feature works
If this change need a document change, I have updated the document
Any dependent changes have been merged

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

morningman · 2020-08-13T14:42:58Z

docs/zh-CN/sql-reference/sql-functions/string-functions/str_replace.md

@@ -0,0 +1,46 @@
+---
+{
+    "title": "str_replace",


In MySQL, this function named replace(), so better to name it as replace()

replace is a keyword has been used by sql syntax analysis.

morningman · 2020-08-13T14:43:12Z

docs/en/sql-reference/sql-functions/string-functions/str_replace.md

+| http://www.baidu.com:                                |
+------------------------------------------------------+
+```
+##keyword


Suggested change

##keyword

## keyword

morningman · 2020-08-13T14:45:02Z

be/src/exprs/string_functions.cpp

+
+StringVal StringFunctions::str_replace(FunctionContext *context, const StringVal &origStr, const StringVal &oldStr, const StringVal &newStr) {
+    if (origStr.is_null || oldStr.is_null || newStr.is_null) {
+        return origStr;


Should return null if any of them is null

morningman · 2020-08-13T14:46:28Z

be/test/exprs/string_functions_test.cpp

+    ASSERT_EQ(StringVal("http://www.baidu.com:9090"),
+        StringFunctions::str_replace(ctx, StringVal("http://www.baidu.com:9090"), StringVal(""), StringVal("8080")));
+    ASSERT_EQ(StringVal("http://www.baidu.com:"),
+        StringFunctions::str_replace(ctx, StringVal("http://www.baidu.com:9090"), StringVal("9090"), StringVal("")));


Better to add more tests to test to corner case, like null, empty string, expr2 longer than expr1, Chinese char, etc...

morningman

LGTM

morningman · 2020-08-16T15:26:45Z

I tested the name replace and it works. So I suggest to rename the str_replace to replace.

You can just patch the following diff file to your code.

diff --git a/fe/fe-core/src/main/cup/sql_parser.cup b/fe/fe-core/src/main/cup/sql_parser.cup
index cd4ea3cdb..7b5e8d2d2 100644
--- a/fe/fe-core/src/main/cup/sql_parser.cup
+++ b/fe/fe-core/src/main/cup/sql_parser.cup
@@ -4592,6 +4592,8 @@ keyword ::=
     {: RESULT = id; :}
     | KW_REPEATABLE:id
     {: RESULT = id; :}
+    | KW_REPLACE:id
+    {: RESULT = id; :}
     | KW_REPLACE_IF_NOT_NULL:id
     {: RESULT = id; :}
     | KW_REPOSITORY:id
diff --git a/gensrc/script/doris_builtins_functions.py b/gensrc/script/doris_builtins_functions.py
index c966b7d8f..ed26fae15 100755
--- a/gensrc/script/doris_builtins_functions.py
+++ b/gensrc/script/doris_builtins_functions.py
@@ -605,7 +605,7 @@ visible_functions = [
             '15FunctionContextENS2_18FunctionStateScopeE'],
     [['concat'], 'VARCHAR', ['VARCHAR', '...'],
             '_ZN5doris15StringFunctions6concatEPN9doris_udf15FunctionContextEiPKNS1_9StringValE'],
-    [['str_replace'], 'VARCHAR', ['VARCHAR', 'VARCHAR', 'VARCHAR'],
+    [['replace'], 'VARCHAR', ['VARCHAR', 'VARCHAR', 'VARCHAR'],
             '_ZN5doris15StringFunctions11str_replaceEPN9doris_udf15FunctionContextERKNS1_9StringValES6_S6_'],
     [['concat_ws'], 'VARCHAR', ['VARCHAR', 'VARCHAR', '...'],
             '_ZN5doris15StringFunctions9concat_wsEPN9doris_udf'

xinghuayu007 · 2020-08-17T06:16:20Z

I tested the name replace and it works. So I suggest to rename the str_replace to replace.

You can just patch the following diff file to your code.

diff --git a/fe/fe-core/src/main/cup/sql_parser.cup b/fe/fe-core/src/main/cup/sql_parser.cup
index cd4ea3cdb..7b5e8d2d2 100644
--- a/fe/fe-core/src/main/cup/sql_parser.cup
+++ b/fe/fe-core/src/main/cup/sql_parser.cup
@@ -4592,6 +4592,8 @@ keyword ::=
     {: RESULT = id; :}
     | KW_REPEATABLE:id
     {: RESULT = id; :}
+    | KW_REPLACE:id
+    {: RESULT = id; :}
     | KW_REPLACE_IF_NOT_NULL:id
     {: RESULT = id; :}
     | KW_REPOSITORY:id
diff --git a/gensrc/script/doris_builtins_functions.py b/gensrc/script/doris_builtins_functions.py
index c966b7d8f..ed26fae15 100755
--- a/gensrc/script/doris_builtins_functions.py
+++ b/gensrc/script/doris_builtins_functions.py
@@ -605,7 +605,7 @@ visible_functions = [
             '15FunctionContextENS2_18FunctionStateScopeE'],
     [['concat'], 'VARCHAR', ['VARCHAR', '...'],
             '_ZN5doris15StringFunctions6concatEPN9doris_udf15FunctionContextEiPKNS1_9StringValE'],
-    [['str_replace'], 'VARCHAR', ['VARCHAR', 'VARCHAR', 'VARCHAR'],
+    [['replace'], 'VARCHAR', ['VARCHAR', 'VARCHAR', 'VARCHAR'],
             '_ZN5doris15StringFunctions11str_replaceEPN9doris_udf15FunctionContextERKNS1_9StringValES6_S6_'],
     [['concat_ws'], 'VARCHAR', ['VARCHAR', 'VARCHAR', '...'],
             '_ZN5doris15StringFunctions9concat_wsEPN9doris_udf'

Thxs!

morningman · 2020-08-17T15:12:16Z

Hi @xinghuayu007 , please let know if you finish modifying this PR, thank you.

xinghuayu007 · 2020-08-18T06:21:30Z

Hi @xinghuayu007 , please let know if you finish modifying this PR, thank you.

I have fixed PR.

morningman

LGTM

replace is an user defined function, which is to replace all old substrings with a new substring in a string, as follow: mysql> select replace("http://www.baidu.com:9090", "9090", ""); +------------------------------------------------------+ | replace('http://www.baidu.com:9090', '9090', '') | +------------------------------------------------------+ | http://www.baidu.com: | +------------------------------------------------------+

udf: replace function

1a15f53

morningman reviewed Aug 13, 2020

View reviewed changes

wangxixu added 2 commits August 14, 2020 09:37

Merge remote-tracking branch 'upstream/master' into str_replace

7d072b9

udf: replace function

391158f

morningman previously approved these changes Aug 14, 2020

View reviewed changes

udf: replace function

5eb52e1

xinghuayu007 dismissed morningman’s stale review via 5eb52e1 August 14, 2020 03:58

morningman self-assigned this Aug 16, 2020

morningman added area/sql/function Issues or PRs related to the SQL functions kind/feature Categorizes issue or PR as related to a new feature. labels Aug 16, 2020

wangxixu added 3 commits August 18, 2020 10:31

Merge remote-tracking branch 'upstream/master' into str_replace

a79ea91

udf: replace function

86f841f

udf: replace function

ade1afa

xinghuayu007 changed the title ~~udf: str_replace function~~ udf: eplace function Aug 18, 2020

xinghuayu007 changed the title ~~udf: eplace function~~ udf: replace function Aug 18, 2020

udf: replace function

1d95a49

morningman approved these changes Aug 18, 2020

View reviewed changes

morningman added the approved Indicates a PR has been approved by one committer. label Aug 18, 2020

morningman merged commit bfb39a2 into apache:master Aug 20, 2020

yangzhg mentioned this pull request Feb 9, 2021

Release Notes 0.14.0 #5374

Closed

yiguolei pushed a commit to yiguolei/incubator-doris that referenced this pull request Nov 4, 2025

Resolve conflict 3.0.7 (apache#4347)

28026c5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

udf: replace function #4347

udf: replace function #4347

Uh oh!

xinghuayu007 commented Aug 13, 2020 •

edited

Loading

Uh oh!

morningman Aug 13, 2020

Uh oh!

xinghuayu007 Aug 14, 2020

Uh oh!

morningman Aug 13, 2020

Uh oh!

morningman Aug 13, 2020

Uh oh!

morningman Aug 13, 2020

Uh oh!

morningman left a comment

Uh oh!

morningman commented Aug 16, 2020

Uh oh!

xinghuayu007 commented Aug 17, 2020

Uh oh!

morningman commented Aug 17, 2020

Uh oh!

xinghuayu007 commented Aug 18, 2020

Uh oh!

morningman left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

udf: replace function #4347

udf: replace function #4347

Uh oh!

Conversation

xinghuayu007 commented Aug 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Types of changes

Checklist

Further comments

Uh oh!

morningman Aug 13, 2020

Choose a reason for hiding this comment

Uh oh!

xinghuayu007 Aug 14, 2020

Choose a reason for hiding this comment

Uh oh!

morningman Aug 13, 2020

Choose a reason for hiding this comment

Uh oh!

morningman Aug 13, 2020

Choose a reason for hiding this comment

Uh oh!

morningman Aug 13, 2020

Choose a reason for hiding this comment

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

morningman commented Aug 16, 2020

Uh oh!

xinghuayu007 commented Aug 17, 2020

Uh oh!

morningman commented Aug 17, 2020

Uh oh!

xinghuayu007 commented Aug 18, 2020

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

xinghuayu007 commented Aug 13, 2020 •

edited

Loading