Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/.vuepress/sidebar/en.js
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,7 @@ module.exports = [
},
"window-function",
"cast",
"digital-masking",
],
},
{
Expand Down
1 change: 1 addition & 0 deletions docs/.vuepress/sidebar/zh-CN.js
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,7 @@ module.exports = [
},
"window-function",
"cast",
"digital-masking",
],
},
{
Expand Down
56 changes: 56 additions & 0 deletions docs/en/sql-reference/sql-functions/digital-masking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
{
"title": "DIGITAL-MASKING",
"language": "en"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# DIGITAL_MASKING

## description

### Syntax

```
digital_masking(digital_number)
```

Alias function, the original function is `concat(left(id,3),'****',right(id,4))`.

Desensitizes the input `digital_number` and returns the result after masking desensitization.

## example

1. Desensitize the cell phone number

```sql
mysql> select digital_masking(13812345678);
+------------------------------+
| digital_masking(13812345678) |
+------------------------------+
| 138****5678 |
+------------------------------+
```

## keyword

DIGITAL_MASKING
Original file line number Diff line number Diff line change
Expand Up @@ -25,50 +25,60 @@ under the License.
-->

# CREATE FUNCTION
##Description
## Description
### Syntax

```
CREATE [AGGREGATE] FUNCTION function_name
(angry type [...])
RETURNS ret_type
[INTERMEDIATE inter_type]
[PROPERTIES ("key" = "value" [, ...]) ]
CREATE [AGGREGATE] [ALIAS] FUNCTION function_name
(arg_type [, ...])
[RETURNS ret_type]
[INTERMEDIATE inter_type]
[WITH PARAMETER(param [,...]) AS origin_function]
[PROPERTIES ("key" = "value" [, ...]) ]
```

### Parameters

>`AGGREGATE`: If this is the case, it means that the created function is an aggregate function, otherwise it is a scalar function.
> `AGGREGATE`: If this is the case, it means that the created function is an aggregate function.
>
>`Function_name`: To create the name of the function, you can include the name of the database. For example: `db1.my_func'.
>
>` arg_type': The parameter type of the function is the same as the type defined at the time of table building. Variable-length parameters can be represented by `,...`. If it is a variable-length type, the type of the variable-length part of the parameters is the same as the last non-variable-length parameter type.
> `ALIAS`: If this is the case, it means that the created function is an alias function.
>
> If the above two items are not present, it means that the created function is a scalar function.
>
> `Function_name`: To create the name of the function, you can include the name of the database. For example: `db1.my_func'.
>
>`ret_type`: Function return type.
> `arg_type`: The parameter type of the function is the same as the type defined at the time of table building. Variable-length parameters can be represented by `,...`. If it is a variable-length type, the type of the variable-length part of the parameters is the same as the last non-variable-length parameter type.
> **NOTICE**: `ALIAS FUNCTION` variable-length parameters are not supported, and there is at least one parameter.
>
> `ret_type`: Required for creating a new function. This parameter is not required if you are aliasing an existing function.
>
>`Inter_type`: A data type used to represent the intermediate stage of an aggregate function.
> `inter_type`: A data type used to represent the intermediate stage of an aggregate function.
>
> `param`: The parameter used to represent the alias function, containing at least one.
>
> `origin_function`: Used to represent the original function corresponding to the alias function.
>
>`properties`: Used to set properties related to this function. Properties that can be set include
> `properties`: Used to set properties related to aggregate function and scalar function. Properties that can be set include
>
> "Object_file": Custom function dynamic library URL path, currently only supports HTTP/HTTPS protocol, this path needs to remain valid throughout the life cycle of the function. This option is mandatory
> "Object_file": Custom function dynamic library URL path, currently only supports HTTP/HTTPS protocol, this path needs to remain valid throughout the life cycle of the function. This option is mandatory
>
> "symbol": Function signature of scalar functions for finding function entries from dynamic libraries. This option is mandatory for scalar functions
> "symbol": Function signature of scalar functions for finding function entries from dynamic libraries. This option is mandatory for scalar functions
>
> "init_fn": Initialization function signature of aggregate function. Necessary for aggregation functions
> "init_fn": Initialization function signature of aggregate function. Necessary for aggregation functions
>
> "update_fn": Update function signature of aggregate function. Necessary for aggregation functions
> "update_fn": Update function signature of aggregate function. Necessary for aggregation functions
>
> "merge_fn": Merge function signature of aggregate function. Necessary for aggregation functions
> "merge_fn": Merge function signature of aggregate function. Necessary for aggregation functions
>
> "serialize_fn": Serialized function signature of aggregate function. For aggregation functions, it is optional, and if not specified, the default serialization function will be used
> "serialize_fn": Serialized function signature of aggregate function. For aggregation functions, it is optional, and if not specified, the default serialization function will be used
>
> "finalize_fn": A function signature that aggregates functions to obtain the final result. For aggregation functions, it is optional. If not specified, the default fetch result function will be used.
> "finalize_fn": A function signature that aggregates functions to obtain the final result. For aggregation functions, it is optional. If not specified, the default fetch result function will be used.
>
> "md5": The MD5 value of the function dynamic link library, which is used to verify that the downloaded content is correct. This option is optional
> "md5": The MD5 value of the function dynamic link library, which is used to verify that the downloaded content is correct. This option is optional
>
> "prepare_fn": Function signature of the prepare function for finding the entry from the dynamic library. This option is optional for custom functions
> "prepare_fn": Function signature of the prepare function for finding the entry from the dynamic library. This option is optional for custom functions
>
> "close_fn": Function signature of the close function for finding the entry from the dynamic library. This option is optional for custom functions
> "close_fn": Function signature of the close function for finding the entry from the dynamic library. This option is optional for custom functions


This statement creates a custom function. Executing this command requires that the user have `ADMIN` privileges.
Expand Down Expand Up @@ -100,12 +110,29 @@ If the `function_name` contains the database name, the custom function will be c

```
CREATE AGGREGATE FUNCTION my_count (BIGINT) RETURNS BIGINT PROPERTIES (
"init_fn"= "_ZN9doris_udf6AddUdfEPNS_15FunctionContextERKNS_6IntValES4_",
"update_fn" = "zn9dorisudf11CountupdateepnsFunctionContexterknsIntvalepnsbigintvale",
"merge_fn" = "zn9dorisudf10CountMergeepnsFunctionContexterknsBigintvaleps2
"finalize_fn" = "zn9dorisudf13CountFinalizepnsFunctionContexterknsBigintvale",
"object_file" = "http://host:port/libudasample.so"
"init_fn"="_ZN9doris_udf9CountInitEPNS_15FunctionContextEPNS_9BigIntValE",
"update_fn"="_ZN9doris_udf11CountUpdateEPNS_15FunctionContextERKNS_6IntValEPNS_9BigIntValE",
"merge_fn"="_ZN9doris_udf10CountMergeEPNS_15FunctionContextERKNS_9BigIntValEPS2_",
"finalize_fn"="_ZN9doris_udf13CountFinalizeEPNS_15FunctionContextERKNS_9BigIntValE",
"object_file"="http://host:port/libudasample.so"
);
```

4. Create a scalar function with variable length parameters

```
CREATE FUNCTION strconcat(varchar, ...) RETURNS varchar properties (
"symbol" = "_ZN9doris_udf6StrConcatUdfEPNS_15FunctionContextERKNS_6IntValES4_",
"object_file" = "http://host:port/libmyStrConcat.so"
);
```

5. Create a custom alias function

```
CREATE ALIAS FUNCTION id_masking(INT) WITH PARAMETER(id)
AS CONCAT(LEFT(id, 3), '****', RIGHT(id, 4));
```

## keyword
CREATE,FUNCTION
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,14 @@ Intermediate Type: NULL
Function Type: Aggregate
Intermediate Type: NULL
Properties: {"object_file":"http://host:port/libudasample.so","finalize_fn":"_ZN9doris_udf13CountFinalizeEPNS_15FunctionContextERKNS_9BigIntValE","init_fn":"_ZN9doris_udf9CountInitEPNS_15FunctionContextEPNS_9BigIntValE","merge_fn":"_ZN9doris_udf10CountMergeEPNS_15FunctionContextERKNS_9BigIntValEPS2_","md5":"37d185f80f95569e2676da3d5b5b9d2f","update_fn":"_ZN9doris_udf11CountUpdateEPNS_15FunctionContextERKNS_6IntValEPNS_9BigIntValE"}
*************************** 3. row ***************************
Signature: id_masking(INT)
Return Type: VARCHAR
Function Type: Alias
Intermediate Type: NULL
Properties: {"parameter":"id","origin_function":"concat(left(`id`, 3), `****`, right(`id`, 4))"}

2 rows in set (0.00 sec)
3 rows in set (0.00 sec)
mysql> show builtin functions in testDb like 'year%';
+---------------+
| Function Name |
Expand Down
56 changes: 56 additions & 0 deletions docs/zh-CN/sql-reference/sql-functions/digital-masking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
{
"title": "DIGITAL-MASKING",
"language": "zh-CN"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# DIGITAL_MASKING

## description

### Syntax

```
digital_masking(digital_number)
```

别名函数,原始函数为 `concat(left(id,3),'****',right(id,4))`。

将输入的 `digital_number` 进行脱敏处理,返回遮盖脱敏后的结果。

## example

1. 将手机号码进行脱敏处理

```sql
mysql> select digital_masking(13812345678);
+------------------------------+
| digital_masking(13812345678) |
+------------------------------+
| 138****5678 |
+------------------------------+
```

## keyword

DIGITAL_MASKING
Original file line number Diff line number Diff line change
Expand Up @@ -25,30 +25,40 @@ under the License.
-->

# CREATE FUNCTION
## description
## Description
### Syntax

```
CREATE [AGGREGATE] FUNCTION function_name
CREATE [AGGREGATE] [ALIAS] FUNCTION function_name
(arg_type [, ...])
RETURNS ret_type
[RETURNS ret_type]
[INTERMEDIATE inter_type]
[WITH PARAMETER(param [,...]) AS origin_function]
[PROPERTIES ("key" = "value" [, ...]) ]
```

### Parameters

> `AGGREGATE`: 如果有此项,表示的是创建的函数是一个聚合函数,否则创建的是一个标量函数。
>
> `AGGREGATE`: 如果有此项,表示的是创建的函数是一个聚合函数。
>
> `ALIAS`:如果有此项,表示的是创建的函数是一个别名函数。
>
> 如果没有上述两项,表示创建的函数是一个标量函数
>
> `function_name`: 要创建函数的名字, 可以包含数据库的名字。比如:`db1.my_func`。
>
> `arg_type`: 函数的参数类型,与建表时定义的类型一致。变长参数时可以使用`, ...`来表示,如果是变长类型,那么变长部分参数的类型与最后一个非变长参数类型一致。
> **注意**:`ALIAS FUNCTION` 不支持变长参数,且至少有一个参数。
>
> `ret_type`: 函数返回类型
> `ret_type`: 对创建新的函数来说,是必填项。如果是给已有函数取别名则可不用填写该参数
>
> `inter_type`: 用于表示聚合函数中间阶段的数据类型。
>
> `properties`: 用于设定此函数相关属性,能够设置的属性包括
> `param`:用于表示别名函数的参数,至少包含一个。
>
> `origin_function`:用于表示别名函数对应的原始函数。
>
> `properties`: 用于设定聚合函数和标量函数相关属性,能够设置的属性包括
>
> "object_file": 自定义函数动态库的URL路径,当前只支持 HTTP/HTTPS 协议,此路径需要在函数整个生命周期内保持有效。此选项为必选项
>
Expand Down Expand Up @@ -97,7 +107,7 @@ CREATE [AGGREGATE] FUNCTION function_name
);
```

2. 创建一个自定义聚合函数
3. 创建一个自定义聚合函数

```
CREATE AGGREGATE FUNCTION my_count (BIGINT) RETURNS BIGINT PROPERTIES (
Expand All @@ -109,14 +119,21 @@ CREATE [AGGREGATE] FUNCTION function_name
);
```

3. 创建一个变长参数的标量函数
4. 创建一个变长参数的标量函数

```
CREATE FUNCTION strconcat(varchar, ...) RETURNS varchar properties (
"symbol" = "_ZN9doris_udf6StrConcatUdfEPNS_15FunctionContextERKNS_6IntValES4_",
"object_file" = "http://host:port/libmyStrConcat.so"
);
```

5. 创建一个自定义别名函数

```
CREATE FUNCTION strconcat(varchar, ...) RETURNS varchar properties (
"symbol" = "_ZN9doris_udf6StrConcatUdfEPNS_15FunctionContextERKNS_6IntValES4_",
"object_file" = "http://host:port/libmyStrConcat.so"
);
```
```
CREATE ALIAS FUNCTION id_masking(INT) WITH PARAMETER(id)
AS CONCAT(LEFT(id, 3), '****', RIGHT(id, 4));
```

## keyword

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,14 @@ Intermediate Type: NULL
Function Type: Aggregate
Intermediate Type: NULL
Properties: {"object_file":"http://host:port/libudasample.so","finalize_fn":"_ZN9doris_udf13CountFinalizeEPNS_15FunctionContextERKNS_9BigIntValE","init_fn":"_ZN9doris_udf9CountInitEPNS_15FunctionContextEPNS_9BigIntValE","merge_fn":"_ZN9doris_udf10CountMergeEPNS_15FunctionContextERKNS_9BigIntValEPS2_","md5":"37d185f80f95569e2676da3d5b5b9d2f","update_fn":"_ZN9doris_udf11CountUpdateEPNS_15FunctionContextERKNS_6IntValEPNS_9BigIntValE"}
*************************** 3. row ***************************
Signature: id_masking(INT)
Return Type: VARCHAR
Function Type: Alias
Intermediate Type: NULL
Properties: {"parameter":"id","origin_function":"concat(left(`id`, 3), `****`, right(`id`, 4))"}

2 rows in set (0.00 sec)
3 rows in set (0.00 sec)
mysql> show builtin functions in testDb like 'year%';
+---------------+
| Function Name |
Expand Down
Loading