diff --git a/docs/en/docs/ecosystem/kyuubi.md b/docs/en/docs/ecosystem/kyuubi.md new file mode 100644 index 00000000000000..618257874ad942 --- /dev/null +++ b/docs/en/docs/ecosystem/kyuubi.md @@ -0,0 +1,115 @@ +--- + +{ +"title": "Kyuubi", +"language": "en" +} + +--- + + + +# Use Kyuubi with Doris + +## Introduction + +[Apache Kyuubi](https://kyuubi.apache.org/) is a distributed and multi-tenant gateway to provide serverless SQL on Data +Warehouses and Lakehouses. +Apache Kyuubi is providing varied protocols like Thrift, Trino and etc., to the engines as Spark, Flink, Hive, JDBC and +etc. +Drois is supported as JDBC data source in Apache Kyuubi. +Apache Kyuubi also provides a serious useful feature with HA, service discovry, +unified authentication, engine lifecycles and etc. + +## Usage + +### Download Apache Kyuubi + +Download Apache Kyuubi from + +Get Apache Kyuubi 1.6.0 or above and extract it to folder。 + +### Config Doris as Kyuubi data source + +- Update Kyuubi configurations in `$KYUUBI_HOME/conf/kyuubi-defaults.conf` + +```properties +kyuubi.engine.type=jdbc +kyuubi.engine.jdbc.type=doris +kyuubi.engine.jdbc.driver.class=com.mysql.cj.jdbc.Driver +kyuubi.engine.jdbc.connection.url=jdbc:mysql://xxx:xxx +kyuubi.engine.jdbc.connection.user=*** +kyuubi.engine.jdbc.connection.password=*** +``` + +| Configuration | Description | +|----------------------------------------|---------------------------------------------------------------| +| kyuubi.engine.type | Enine Type, specify to `jdbc` | +| kyuubi.engine.jdbc.type | JDBC service type, specify to `doris` | +| kyuubi.engine.jdbc.connection.url | JDBC url to Doris FE | +| kyuubi.engine.jdbc.connection.user | JDBC username | +| kyuubi.engine.jdbc.connection.password | JDBC password | +| kyuubi.engine.jdbc.driver.class | JDBC driver class name, specify to `com.mysql.cj.jdbc.Driver` | + +- For other configuration in Apache Kyuubi, please refer + to [Apache Kyuubi Configuration Docs](https://kyuubi.readthedocs.io/en/master/deployment/settings.html) + +### Add MySQL JDBC Driver + +Copy the Mysql JDBC Driver `mysql-connector-j-8.X.X.jar` to `$KYUUBI_HOME/incubating-bin/externals/engines/jdbc` 目录下。 + +### Start Kyuubi Server + +Run `$KYUUBI_HOME/bin/kyuubi run`. +After started, port 10009 by default is listened by Kyuubi Server with Thrift protocol。 + +## Example + +The fowllowing example shows basic example of querying Doris with Kyuubi with beeline CLI in thift protocol. + +### Connect to Kyuubi with Beeline + +```shell +$ ./beeline -u "jdbc:hive2://xxxx:10009/" +``` + +### Execute Query to Kyuubi + +Execute query statement `select * from demo.expamle_tbl;` with query results returned. + +```shell +0: jdbc:hive2://xxxx:10009/> select * from demo.example_tbl; + +2023-03-07 09:29:14.771 INFO org.apache.kyuubi.operation.ExecuteStatement: Processing anonymous's query[bdc59dd0-ceea-4c02-8c3a-23424323f5db]: PENDING_STATE -> RUNNING_STATE, statement: +select * from demo.example_tbl +2023-03-07 09:29:14.786 INFO org.apache.kyuubi.operation.ExecuteStatement: Query[bdc59dd0-ceea-4c02-8c3a-23424323f5db] in FINISHED_STATE +2023-03-07 09:29:14.787 INFO org.apache.kyuubi.operation.ExecuteStatement: Processing anonymous's query[bdc59dd0-ceea-4c02-8c3a-23424323f5db]: RUNNING_STATE -> FINISHED_STATE, time taken: 0.015 seconds ++----------+-------------+-------+------+------+------------------------+-------+-----------------+-----------------+ +| user_id | date | city | age | sex | last_visit_date | cost | max_dwell_time | min_dwell_time | ++----------+-------------+-------+------+------+------------------------+-------+-----------------+-----------------+ +| 10000 | 2017-10-01 | 北京 | 20 | 0 | 2017-10-01 07:00:00.0 | 70 | 10 | 2 | +| 10001 | 2017-10-01 | 北京 | 30 | 1 | 2017-10-01 17:05:45.0 | 4 | 22 | 22 | +| 10002 | 2017-10-02 | 上海 | 20 | 1 | 2017-10-02 12:59:12.0 | 400 | 5 | 5 | +| 10003 | 2017-10-02 | 广州 | 32 | 0 | 2017-10-02 11:20:00.0 | 60 | 11 | 11 | +| 10004 | 2017-10-01 | 深圳 | 35 | 0 | 2017-10-01 10:00:15.0 | 200 | 3 | 3 | +| 10004 | 2017-10-03 | 深圳 | 35 | 0 | 2017-10-03 10:20:22.0 | 22 | 6 | 6 | ++----------+-------------+-------+------+------+------------------------+-------+-----------------+-----------------+ +6 rows selected (0.068 seconds) +``` diff --git a/docs/sidebars.json b/docs/sidebars.json index 363a7da352915c..dcc78f198212a6 100644 --- a/docs/sidebars.json +++ b/docs/sidebars.json @@ -224,6 +224,7 @@ "ecosystem/spark-doris-connector", "ecosystem/flink-doris-connector", "ecosystem/datax", + "ecosystem/kyuubi", "ecosystem/mysql-to-doris", "ecosystem/logstash", "ecosystem/plugin-development-manual", diff --git a/docs/zh-CN/docs/ecosystem/kyuubi.md b/docs/zh-CN/docs/ecosystem/kyuubi.md new file mode 100644 index 00000000000000..4e00f04f2235d8 --- /dev/null +++ b/docs/zh-CN/docs/ecosystem/kyuubi.md @@ -0,0 +1,114 @@ +--- + +{ +"title": "Kyuubi", +"language": "zh-CN" +} + +--- + + + +# 通过 Kyuubi 连接 Doris + +## 介绍 + +[Apache Kyuubi](https://kyuubi.apache.org/)，一个分布式和多租户网关，用于在 Lakehouse 上提供 Serverless +SQL，可连接包括Spark、Flink、Hive、JDBC等引擎，并对外提供Thrift、Trino等接口协议供灵活对接。 +其中Apache Kyuubi实现了JDBC Engine并支持Doris方言，并可用于对接Doris作为数据源。 +Apache Kyuubi可提供高可用、服务发现、租户隔离、统一认证、生命周期等一系列特性。 + +## 下载 Apache Kyuubi + +## 配置方法 + +### 下载 Apache Kyuubi + +从官网下载Apache Kyuubi 1.6.0或以上版本的安装包后解压。 + +Apache Kyuubi 下载地址： + +### 配置Doris作为Kyuubi数据源 + +- 修改配置文件 `$KYUUBI_HOME/conf/kyuubi-defaults.conf` + +```properties +kyuubi.engine.type=jdbc +kyuubi.engine.jdbc.type=doris +kyuubi.engine.jdbc.driver.class=com.mysql.cj.jdbc.Driver +kyuubi.engine.jdbc.connection.url=jdbc:mysql://xxx:xxx +kyuubi.engine.jdbc.connection.user=*** +kyuubi.engine.jdbc.connection.password=*** +``` + +| 配置项 | 说明 | +|----------------------------------------|-----------------------------------------------| +| kyuubi.engine.type | 引擎类型。请使用jdbc | +| kyuubi.engine.jdbc.type | jdbc服务类型。这里请指定为doris | +| kyuubi.engine.jdbc.connection.url | jdbc 服务连接。这里请指定 Doris FE 上的 mysql server 连接地址 | +| kyuubi.engine.jdbc.connection.user | jdbc 服务用户名 | +| kyuubi.engine.jdbc.connection.password | jdbc 服务密码 | +| kyuubi.engine.jdbc.driver.class | 连接jdbc服务使用的驱动类名。请使用com.mysql.cj.jdbc.Driver | + +- 其他相关配置参考 [Apache Kyuubi配置说明](https://kyuubi.readthedocs.io/en/master/deployment/settings.html) + +### 添加MySQL驱动 + +添加 Mysql JDB C驱动 `mysql-connector-j-8.X.X.jar` 到 `$KYUUBI_HOME/incubating-bin/externals/engines/jdbc` 目录下。 + +### 启动 Kyuubi 服务 + +`$KYUUBI_HOME/bin/kyuubi run` +启动后，Kyuubi默认监听10009端口提供Thrift协议。 + +## 使用方法 + +以下例子展示通过Apache Kyuubi的beeline工具经Thrift协议查询Doris。 + +### 建立连接 + +```shell +$ ./beeline -u "jdbc:hive2://xxxx:10009/" + +``` + +### 执行查询 + +执行查询语句 `select * from demo.expamle_tbl;` 并得到结果。 + +```shell +0: jdbc:hive2://xxxx:10009/> select * from demo.example_tbl; + +2023-03-07 09:29:14.771 INFO org.apache.kyuubi.operation.ExecuteStatement: Processing anonymous's query[bdc59dd0-ceea-4c02-8c3a-23424323f5db]: PENDING_STATE -> RUNNING_STATE, statement: +select * from demo.example_tbl +2023-03-07 09:29:14.786 INFO org.apache.kyuubi.operation.ExecuteStatement: Query[bdc59dd0-ceea-4c02-8c3a-23424323f5db] in FINISHED_STATE +2023-03-07 09:29:14.787 INFO org.apache.kyuubi.operation.ExecuteStatement: Processing anonymous's query[bdc59dd0-ceea-4c02-8c3a-23424323f5db]: RUNNING_STATE -> FINISHED_STATE, time taken: 0.015 seconds ++----------+-------------+-------+------+------+------------------------+-------+-----------------+-----------------+ +| user_id | date | city | age | sex | last_visit_date | cost | max_dwell_time | min_dwell_time | ++----------+-------------+-------+------+------+------------------------+-------+-----------------+-----------------+ +| 10000 | 2017-10-01 | 北京 | 20 | 0 | 2017-10-01 07:00:00.0 | 70 | 10 | 2 | +| 10001 | 2017-10-01 | 北京 | 30 | 1 | 2017-10-01 17:05:45.0 | 4 | 22 | 22 | +| 10002 | 2017-10-02 | 上海 | 20 | 1 | 2017-10-02 12:59:12.0 | 400 | 5 | 5 | +| 10003 | 2017-10-02 | 广州 | 32 | 0 | 2017-10-02 11:20:00.0 | 60 | 11 | 11 | +| 10004 | 2017-10-01 | 深圳 | 35 | 0 | 2017-10-01 10:00:15.0 | 200 | 3 | 3 | +| 10004 | 2017-10-03 | 深圳 | 35 | 0 | 2017-10-03 10:20:22.0 | 22 | 6 | 6 | ++----------+-------------+-------+------+------+------------------------+-------+-----------------+-----------------+ +6 rows selected (0.068 seconds) +```