Skip to content

[Proposal] Support Limit Clause Push Down For ODBC Table And MySQL Table #4706

@HappenLee

Description

@HappenLee

Motivation

At present, Doris do not push down limit operation to Exteranl Table Engine.It may serious performance problems in some scenarios. For example, this is a Exteranl Table of Mysql:

CREATE EXTERNAL TABLE `baseall_mysql` (
  `k1` tinyint(4) NULL COMMENT "",
  `k2` smallint(6) NULL COMMENT "",
  `k3` int(11) NULL COMMENT "",
  `k4` bigint(20) NULL COMMENT "",
  `k5` decimal(9, 3) NULL COMMENT "",
  `k6` char(5) NULL COMMENT "",
  `k10` date NULL COMMENT "",
  `k11` datetime NULL COMMENT "",
  `k7` varchar(20) NULL COMMENT "",
  `k8` double NULL COMMENT "",
  `k9` float NULL COMMENT ""
) ENGINE=MYSQL
COMMENT "MYSQL"
PROPERTIES (
"host" = "192.168.1.1",
"port" = "8086",
"user" = "root",
"password" = "123456",
"database" = "test",
"table" = "base"
);

This table have 60000 row in external table.

We do query like:

select * from baseall_mysql limit 10;

Now Doris do not push down limit clause of mysql table, so Doris fetch all row in baseall_mysql and get top 10 of the table.it is cost unnecessary network and mem in MySQL Scan Node / ODBC Scan Node.

In my test:
this query cost 2.1s but push down the limit clause, it only take 0.21s.

So we need to support limit clause push down for ODBC Table and MySQL Table.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions