Skip to content

[Proto]: Serialization support for AsyncFuncExec#19118

Merged
Jefffrey merged 12 commits intoapache:mainfrom
spiceai:mach/serde-async-func-exec
Dec 10, 2025
Merged

[Proto]: Serialization support for AsyncFuncExec#19118
Jefffrey merged 12 commits intoapache:mainfrom
spiceai:mach/serde-async-func-exec

Conversation

@mach-kernel
Copy link
Contributor

Which issue does this PR close?

Closes #19112

Rationale for this change

Use async functions with Ballista

What changes are included in this PR?

  • New AsyncFuncExecNode proto definition
  • to/from glue
  • A roundtrip test

Are these changes tested?

n/a

Are there any user-facing changes?

n/a

@github-actions github-actions bot added proto Related to proto crate physical-plan Changes to the physical-plan crate labels Dec 5, 2025
Copy link
Contributor

@Jefffrey Jefffrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for raising this, left some suggestions for simplifications

Comment on lines 2355 to 2361
let logical_plan = ctx
.state()
.create_logical_plan("select test_async_udf(1)")
.await?;
let physical_plan = ctx.state().create_physical_plan(&logical_plan).await?;
let codec = TestAsyncUDFPhysicalCodec {};
roundtrip_test_and_return(physical_plan, &ctx, &codec)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let logical_plan = ctx
.state()
.create_logical_plan("select test_async_udf(1)")
.await?;
let physical_plan = ctx.state().create_physical_plan(&logical_plan).await?;
let codec = TestAsyncUDFPhysicalCodec {};
roundtrip_test_and_return(physical_plan, &ctx, &codec)?;
let plan = ctx
.sql("select test_async_udf(1)")
.await?
.create_physical_plan()
.await?;
roundtrip_test_with_context(plan, &ctx)?;

Simplify the test a little

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but still need the codec.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested using roundtrip_test_and_return and it seems to work fine without the codec? Perhaps I'm missing something here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake -- I was using roundtrip_test without the context (which did not have the UDF registered). Updated 😄!

@mach-kernel mach-kernel requested a review from Jefffrey December 8, 2025 20:16
@Jefffrey Jefffrey added this pull request to the merge queue Dec 10, 2025
Merged via the queue into apache:main with commit ca67edc Dec 10, 2025
34 checks passed
@Jefffrey
Copy link
Contributor

Thanks @mach-kernel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support ser/de of AsyncFuncExec

2 participants