Skip to content

Conversation

@andygrove
Copy link
Member

@andygrove andygrove commented May 22, 2022

Which issue does this PR close?

Closes #2345

Rationale for this change

I would like the ability to have projects just depend on datafusion-sql if they are using DataFusion as a SQL parser + query planner and are not using it for execution.

What changes are included in this PR?

  • Moved code around without functional changes mostly
  • API change to use TableSource instead of TableProvider in the schema provider trait that the SQL planner uses
  • Modified a few tests to just test the SQL planner and not the optimizer rules (which are not available to this crate)
  • Updated release notes and scripts

Here is the new crate dependency diagram:

crate-deps

Are there any user-facing changes?

Yes, API change and new crate.

@andygrove andygrove added the api change Changes the API exposed to users of the crate label May 22, 2022
@andygrove andygrove self-assigned this May 22, 2022
quick_test(sql, expected);
}

#[test]
fn test_offset_after_limit_with_limit_push() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't be testing optimizer rules here, just the SQL query planner

@andygrove andygrove marked this pull request as ready for review May 22, 2022 16:07
@andygrove andygrove changed the title WIP: Move SQL Query Planner into new datafusion-sql crate Move SQL Query Planner into new datafusion-sql crate May 22, 2022
@github-actions github-actions bot added the development-process Related to development process of DataFusion label May 22, 2022
https://crates.io/crates/datafusion-physical-expr/8.0.0
https://crates.io/crates/datafusion-proto/8.0.0
https://crates.io/crates/datafusion-data-access/8.0.0
https://crates.io/crates/ballista/0.7.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing ballista here is not related to this PR but was missed earlier

@andygrove andygrove requested review from alamb and yjshen May 23, 2022 13:44
Copy link
Member

@yjshen yjshen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@andygrove andygrove merged commit dde6f71 into apache:master May 23, 2022
@andygrove andygrove deleted the datafusion-sql branch May 23, 2022 16:13
@alamb
Copy link
Contributor

alamb commented May 23, 2022

I didn't review the code of this PR but I like where it is heading -- thank you @andygrove

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api change Changes the API exposed to users of the crate development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move SQL query planning to new crate

3 participants