Skip to content

Conversation

@alamb
Copy link
Contributor

@alamb alamb commented Nov 1, 2024

Which issue does this PR close?

Part of #12591

Rationale for this change

Before LexOrdering and LexRequirement were actual structs (thanks to @ngli-me and @berkaysynnada on #13146!!!), we couldn't put the conversion between LexOrdering and LexRequirement on to the actual structures and thus the conversion is done via free functions on PhysicalSortRequirements

This is confusing for several reasons:

  • The conversion is on another struct (so when Rust programmers go to look for a From conversion they aren't there)
  • The conversions hide a clone when ideomatic Rust mostly prefers explict cloning when it is occuring

This structure bothered me for a long time, so it is time to scratch an itch now that @ngli-me started the ball rolling

What changes are included in this PR?

  1. Deprecate PhysicalSortRequirement::from_sort_exprs and PhysicalSortRequirement::to_sort_exprs
  2. Add LexOrdering::from_lex_requirement and LexOrdering::from_lex_ordering and From impls
  3. Update code

Are these changes tested?

By existint CI

Are there any user-facing changes?

SOme methods are deprecated with hints about what to use instead

@alamb alamb marked this pull request as draft November 1, 2024 19:36
@github-actions github-actions bot added physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) proto Related to proto crate functions Changes to functions implementation labels Nov 1, 2024
@alamb alamb force-pushed the alamb/deprecate_conversions branch from 50ded20 to 79278f8 Compare November 1, 2024 19:39
@github-actions github-actions bot removed sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Nov 1, 2024
vec![Some(PhysicalSortRequirement::from_sort_exprs(
self.expr.iter(),
))]
vec![Some(LexRequirement::from(self.expr.clone()))]
Copy link
Contributor Author

@alamb alamb Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty good example of the potential improvement -- it is now clear this function is returning a LexRequirement and that it clones the underlying LexOrdering structure

This was still happening on main but the conversion was hidden

shift_right_required(parent_required, right_offset)?;
let new_right_required_expr =
PhysicalSortRequirement::to_sort_exprs(new_right_required);
let new_right_required_expr = LexOrdering::from(new_right_required);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this structure makes it much clearer what is going on

@alamb alamb marked this pull request as ready for review November 1, 2024 19:42
@alamb alamb requested a review from berkaysynnada November 1, 2024 19:42
// sort and already satisfied by the given ordering
let child_eq_properties = child.equivalence_properties();
let sort_req = PhysicalSortRequirement::from_sort_exprs(sort_plan.expr());
let sort_req = LexRequirement::from(sort_plan.expr().clone());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, there is no more cloning going on than previously, just now the clone is explicit

Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, the code is clearly improving. Thank you @alamb.

@alamb alamb merged commit 6a02384 into apache:main Nov 6, 2024
@alamb
Copy link
Contributor Author

alamb commented Nov 6, 2024

Thank you for the review @berkaysynnada

@alamb alamb deleted the alamb/deprecate_conversions branch November 6, 2024 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate optimizer Optimizer rules physical-expr Changes to the physical-expr crates proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants