Skip to content

[regression](catalog)Add test case of paimon complex type #25834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morningman merged 3 commits into from
Nov 1, 2023
Merged

[regression](catalog)Add test case of paimon complex type #25834

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b263557
[regression](catalog)Add test case of paimon complex type
DongLiang-0 Oct 25, 2023
4251ac6
add array nested test
DongLiang-0 Oct 26, 2023
a6484ef
fix
DongLiang-0 Oct 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions docs/en/docs/lakehouse/multi-catalog/paimon.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ under the License.
## Instructions for use

1. When data in hdfs,need to put core-site.xml, hdfs-site.xml and hive-site.xml in the conf directory of FE and BE. First read the hadoop configuration file in the conf directory, and then read the related to the environment variable `HADOOP_CONF_DIR` configuration file.
2. The currently adapted version of the payment is 0.4.0
2. The currently adapted version of the payment is 0.5.0

## Create Catalog

Expand Down Expand Up @@ -64,7 +64,7 @@ CREATE CATALOG `paimon_hdfs` PROPERTIES (

> Note that.
>
> user need download [paimon-s3-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.4.0-incubating/paimon-s3-0.4.0-incubating.jar)
> user need download [paimon-s3-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.5.0-incubating/paimon-s3-0.5.0-incubating.jar)
>
> Place it in directory ${DORIS_HOME}/be/lib/java_extensions/preload-extensions and restart be
>
Expand All @@ -85,7 +85,7 @@ CREATE CATALOG `paimon_s3` PROPERTIES (

>Note that.
>
> user need download [paimon-oss-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.4.0-incubating/paimon-oss-0.4.0-incubating.jar)
> user need download [paimon-oss-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.5.0-incubating/paimon-oss-0.5.0-incubating.jar)
> Place it in directory ${DORIS_HOME}/be/lib/java_extensions/preload-extensions and restart be


Expand Down Expand Up @@ -120,4 +120,21 @@ CREATE CATALOG `paimon_hms` PROPERTIES (

## Column Type Mapping

Same as that in Hive Catalogs. See the relevant section in [Hive](./hive.md).
| Paimon Data Type | Doris Data Type | Comment |
|---------------------------------------|---------------------------|-----------|
| BooleanType | Boolean | |
| TinyIntType | TinyInt | |
| SmallIntType | SmallInt | |
| IntType | Int | |
| FloatType | Float | |
| BigIntType | BigInt | |
| DoubleType | Double | |
| VarCharType | VarChar | |
| CharType | Char | |
| DecimalType(precision, scale) | Decimal(precision, scale) | |
| TimestampType,LocalZonedTimestampType | DateTime | |
| DateType | Date | |
| MapType | Map | Support Map nesting |
| ArrayType | Array | Support Array nesting |
| VarBinaryType, BinaryType | Binary | |

25 changes: 21 additions & 4 deletions docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ under the License.
## 使用须知

1. 数据放在hdfs时，需要将 core-site.xml，hdfs-site.xml 和 hive-site.xml 放到 FE 和 BE 的 conf 目录下。优先读取 conf 目录下的 hadoop 配置文件，再读取环境变量 `HADOOP_CONF_DIR` 的相关配置文件。
2. 当前适配的paimon版本为0.4.0
2. 当前适配的paimon版本为0.5.0

## 创建 Catalog

Expand Down Expand Up @@ -64,7 +64,7 @@ CREATE CATALOG `paimon_hdfs` PROPERTIES (

> 注意：
>
> 用户需要手动下载[paimon-s3-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.4.0-incubating/paimon-s3-0.4.0-incubating.jar)
> 用户需要手动下载[paimon-s3-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.5.0-incubating/paimon-s3-0.5.0-incubating.jar)

> 放在${DORIS_HOME}/be/lib/java_extensions/preload-extensions目录下并重启be。
>
Expand All @@ -85,7 +85,7 @@ CREATE CATALOG `paimon_s3` PROPERTIES (

>注意：
>
> 用户需要手动下载[paimon-oss-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.4.0-incubating/paimon-oss-0.4.0-incubating.jar)
> 用户需要手动下载[paimon-oss-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.5.0-incubating/paimon-oss-0.5.0-incubating.jar)
> 放在${DORIS_HOME}/be/lib/java_extensions/preload-extensions目录下并重启be

```sql
Expand Down Expand Up @@ -119,5 +119,22 @@ CREATE CATALOG `paimon_hms` PROPERTIES (

## 列类型映射

和 Hive Catalog 基本一致，可参阅 [Hive Catalog](./hive.md) 中 **列类型映射** 一节。
| Paimon Data Type | Doris Data Type | Comment |
|---------------------------------------|---------------------------|-----------|
| BooleanType | Boolean | |
| TinyIntType | TinyInt | |
| SmallIntType | SmallInt | |
| IntType | Int | |
| FloatType | Float | |
| BigIntType | BigInt | |
| DoubleType | Double | |
| VarCharType | VarChar | |
| CharType | Char | |
| DecimalType(precision, scale) | Decimal(precision, scale) | |
| TimestampType,LocalZonedTimestampType | DateTime | |
| DateType | Date | |
| MapType | Map | 支持Map嵌套 |
| ArrayType | Array | 支持Array嵌套 |
| VarBinaryType, BinaryType | Binary | |


Loading