[Bug] SelectNode.cardinality < 0 cause query exception "Unexpected exception: null"

[ccoffline]

Describe the bug
When using select condition like -1=-1 may cause errCode = 2, detailMessage = Unexpected exception: null.
This is due toSelectNode.cardinality computing error.

https://github.com/apache/incubator-doris/blob/81aa1c599def3c3f0d9cd80469fd06393ea764e7/fe/src/main/java/org/apache/doris/planner/SelectNode.java#L63-L73

This cardinality may be less than 0.

To Reproduce
Steps to reproduce the behavior:

1. Create a table

CREATE TABLE `table1` (
  `city_id` bigint(20) NULL,
  `tag_id` bigint(20) NULL,
  `dt` bigint(20) NULL,
  `num` bigint(20) NULL
) ENGINE=OLAP
DUPLICATE KEY(`city_id`, `tag_id`, `dt`)
COMMENT "OLAP"
PARTITION BY RANGE(`dt`)
(PARTITION p20200102 VALUES [("20200101"), ("20200102")),
PARTITION p20200103 VALUES [("20200102"), ("20200103")))
DISTRIBUTED BY HASH(`city_id`) BUCKETS 5
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);

2. Insert some data

INSERT INTO table1 VALUES (1, 2, 20200102, 3);

You may need to wait for the cardinality to be > 0 (I still dont know how the cardinality works).

2. Launch a query

SELECT rank, tag_id
FROM (
    SELECT row_number() OVER (ORDER BY num DESC) AS rank, tag_id
    FROM table1
    ) t
WHERE -1=-1 or tag_id in (-1)
LIMIT 10
;

When the table is not empty (maybe cardinality means row number?), cardinality will be > 0, computeSelectivity() < 0, then the error occur.

I tried to remove some elements and this is the minimum sql to reproduce the bug.

Could someone explain what doe cardinality and selectivity means?
Is Preconditions.checkState(cardinality >= 0); necessary?

[ccoffline]

-1=-1 or other True expr will turn into a BoolLiteral which selectivity = -1.
tag_id in (-1) will turn into a InPredicate which selectivity = 0.1.
So in their father node CompoundPredicate, selectivity will be set to -1.

https://github.com/apache/incubator-doris/blob/81aa1c599def3c3f0d9cd80469fd06393ea764e7/fe/src/main/java/org/apache/doris/analysis/CompoundPredicate.java#L106-L115

If remove any of these expresstions, SelectNode.computeStats will not be executed.

[HappenLee]

Hi，@ccoffline。Thanks for reporting this problem。You can theat cardinality like row number. Now Doris only use cardinality to chose join type, it will be widely used in the future. Queries should not be affected， so we will simplely fix the problem.

[ccoffline]

thx guys

[ccoffline]

This happens again when a CaseExpr didn't set its selectivity.

It seems a lot of Exprs haven't set selectivity properly. And they will all cause this computeStats error.

I will count all of these Exprs later and put a list here.

[ccoffline]

Most of Exprs haven't set selectivity so I'll just list those that have.

• LikePredicate
• InPredicate
• IsNullPredicate
• BinaryPredicate
• CompoundPredicate
• BoolLiteral that Set BoolLiteral's selectivity #4590 has just added
• SlotRef

There are lots that need to modify. Since queries should not be affected, I'm thinking that cardinality of SelectNode should just set to -1 when computeSelectivity() is not correct instead of hard preconditions that produce unknown error.

[liutang123]

Give your PR linked this issue please.