Skip to content

Incorrect results in COUNT(*) queries involving LIMIT #8048

@msirek

Description

@msirek

Describe the bug

While testing #8038, I ran into some incorrect results cases in LIMIT queries.

To Reproduce

❯ CREATE TABLE IF NOT EXISTS t1 (a INT) AS VALUES(1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
0 rows in set. Query took 0.009 seconds.

❯ EXPLAIN
SELECT COUNT(*) FROM (SELECT a FROM t1 WHERE a > 3 LIMIT 3 OFFSET 6);
+---------------+---------------------------------------------------------------+
| plan_type     | plan                                                          |
+---------------+---------------------------------------------------------------+
| logical_plan  | Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1)) AS COUNT(*)]] |
|               |   Limit: skip=6, fetch=3                                      |
|               |     Filter: t1.a > Int32(3)                                   |
|               |       TableScan: t1 projection=[a]                            |
| physical_plan | ProjectionExec: expr=[9 as COUNT(*)]                          |
|               |   EmptyExec: produce_one_row=true                             |
|               |                                                               |
+---------------+---------------------------------------------------------------+
2 rows in set. Query took 0.015 seconds.

-- The correct answer is 1.SELECT COUNT(*) FROM (SELECT a FROM t1 WHERE a > 3 LIMIT 3 OFFSET 6);
+----------+
| COUNT(*) |
+----------+
| 9        |
+----------+

❯ CREATE EXTERNAL TABLE sales(customer_id VARCHAR, revenue BIGINT) STORED AS CSV location '/home/ms/git/arrow-datafusion/datafusion/core/tests/data/customer.csv';
0 rows in set. Query took 0.002 seconds.

-- The correct answer is 1.SELECT COUNT(*) FROM (SELECT customer_id FROM sales LIMIT 10000000000);
+-------------+
| COUNT(*)    |
+-------------+
| 10000000000 |
+-------------+
1 row in set. Query took 0.009 seconds.

❯ explain  SELECT COUNT(*) FROM (SELECT customer_id FROM sales LIMIT 10000000000);
+---------------+---------------------------------------------------------------+
| plan_type     | plan                                                          |
+---------------+---------------------------------------------------------------+
| logical_plan  | Aggregate: groupBy=[[]], aggr=[[COUNT(UInt8(1)) AS COUNT(*)]] |
|               |   Limit: skip=0, fetch=10000000000                            |
|               |     TableScan: sales projection=[], fetch=10000000000         |
| physical_plan | ProjectionExec: expr=[10000000000 as COUNT(*)]                |
|               |   EmptyExec: produce_one_row=true                             |
|               |                                                               |
+---------------+---------------------------------------------------------------+
2 rows in set. Query took 0.015 seconds.

Expected behavior

Correct results from query execution

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions