fix: Add additional required expression for natural join

[Lordworms]

Which issue does this PR close?

Closes #11635

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[alamb]

Thanks @Lordworms -- this code looks reasonable to me. I had some suggestions / requests for some more comments.

Maybe @jonahgao has ideas of how to improve this too

[jonahgao]

I checked that the final logical plan is correct, but the calling of expand_wildcard in wrap_projection_for_join_if_necessary breaks it. expand_wildcard removes duplicate columns in the using join, but this column is needed by the parent plan.

The use of expand_wildcard was introduced in #4443. I speculate that it was a refactor of the following old implementation.

let mut projection = vec![Expr::Wildcard];
        projection.extend(expr_join_keys.into_iter());
        LogicalPlanBuilder::from(input)
            .project(projection)?
            .build()?

Its purpose should be to re-project all the outputs of the input plan.

So I think another simpler fix might be to replace expand_wildcard with this.

[Lordworms]

I checked that the final logical plan is correct, but the calling of expand_wildcard in wrap_projection_for_join_if_necessary breaks it. expand_wildcard removes duplicate columns in the using join, but this column is needed by the parent plan.

The use of expand_wildcard was introduced in #4443. I speculate that it was a refactor of the following old implementation.

let mut projection = vec![Expr::Wildcard];
        projection.extend(expr_join_keys.into_iter());
        LogicalPlanBuilder::from(input)
            .project(projection)?
            .build()?

Its purpose should be to re-project all the outputs of the input plan.

So I think another simpler fix might be to replace expand_wildcard with this.
expand_wildcard

I agree, It's is much simpler, I was thinking about how to add missing expressions that are excluded by expand_wildcard but haven't thought about replace it directly. Thanks for this.

[jonahgao]

Looks good to me!

[alamb]

Thank you @Lordworms and @jonahgao -- this is a beautifully simple PR now. 🏆

[alamb]

😍 that is beautiful.