Move tpch tests into sqllogictest.

[liurenjie1024]

Is your feature request related to a problem or challenge?

Currently for tpch tests, there are for tests in tpch benchmark:

1. Verify generated plan
2. Verify empty run
3. Verify query result with some data
4. Verify plan serialization/deserialization

Describe the solution you'd like

To better improve maintainability, I would like to move 1-3 to sqllogictest.

Describe alternatives you've considered

No response

Additional context

No response

[liurenjie1024]

I didn't realize tht it already exists, close it.

[alamb]

This is an awesome idea -- thank you @liurenjie1024

When you say "already exists" what do you mean?

There is still code in benchmarks to run tpch / expected queries: https://github.com/apache/arrow-datafusion/tree/main/benchmarks/expected-plans

Can we maybe move that code into sqllogictests so it is easier to discover and maintain?

[liurenjie1024]

I see this https://github.com/apache/arrow-datafusion/blob/main/datafusion/core/tests/sqllogictests/test_files/tpch.slt, but it seems that it's not enough, will go on with it.

[alamb]

Thank you @liurenjie1024

[mingmwang]

@alamb @liurenjie1024

I do not think moving the tpch plan verification into sqllogictest is a good idea, because the plan is usually quite complex and It is very hard to compare the plan and fix the change part in the slt files.

[mingmwang]

An example:
a752ee8

[liurenjie1024]

@alamb @liurenjie1024

I do not think moving the tpch plan verification into sqllogictest is a good idea, because the plan is usually quite complex and It is very hard to compare the plan and fix the change part in the slt files.

I think the complexity of comparing is caused by text plan representation. As with fixing the change part, you can run --complete mode, which will fix slt result automatically.

[alamb]

I think the complexity of comparing is caused by text plan representation. As with fixing the change part, you can run --complete mode, which will fix slt result automatically.

I agree with @liurenjie1024 that it is the textual representation that makes comparison hard, not the .slt files.

@mingmwang one of the big benefits in my mind to slt files is that they are easy to update.

For example, you could create the change in a752ee8 by running

cargo test -p datafusion --test sqllogictest -- tpch --complete

And then you can review the changes and check them in

(Complete docs are in https://github.com/apache/arrow-datafusion/blob/main/datafusion/core/tests/sqllogictests/README.md)

Does that work for you? If not, is there something that would make it easier for you?