Skip to content

[GLUTEN-7652][VL] Support binary as string#9325

Merged
wForget merged 10 commits intoapache:mainfrom
wForget:GLUTEN-7652
Apr 16, 2025
Merged

[GLUTEN-7652][VL] Support binary as string#9325
wForget merged 10 commits intoapache:mainfrom
wForget:GLUTEN-7652

Conversation

@wForget
Copy link
Copy Markdown
Member

@wForget wForget commented Apr 15, 2025

What changes were proposed in this pull request?

Velox supports binary as string of parquet after facebookincubator/velox#10399, we need to pass hive columns type to HiveTableHandle

Fixes: #7652

How was this patch tested?

added unit test

@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Apr 15, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

#7652

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

@wForget wForget marked this pull request as draft April 15, 2025 05:59
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

wForget
wForget commented Apr 15, 2025
View reviewed changes
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

@github-actions

This comment was marked as duplicate.

Sign in to view
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2025

Run Gluten Clickhouse CI on x86

@wForget wForget marked this pull request as ready for review April 15, 2025 09:50
@wForget wForget changed the title [GLUTEN-7652][VL] Set data columns for HiveTableHandle [GLUTEN-7652][VL] Support binary as string Apr 15, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 16, 2025

Run Gluten Clickhouse CI on x86

Yohahaha
Yohahaha approved these changes Apr 16, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@Yohahaha Yohahaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@wForget
Copy link
Copy Markdown
Member Author

wForget commented Apr 16, 2025

There is a related failure, I will improve uint test to be compatible with spark 3.2.

- test binary as string *** FAILED ***
  org.apache.spark.sql.AnalysisException: Undefined function: 'to_binary'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 0
  at org.apache.spark.sql.catalyst.analysis.Analyzer$LookupFunctions$$anonfun$apply$25.$anonfun$applyOrElse$114(Analyzer.scala:2115)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 16, 2025

Run Gluten Clickhouse CI on x86

@wForget wForget requested a review from Yohahaha April 16, 2025 10:09
@wForget
Copy link
Copy Markdown
Member Author

wForget commented Apr 16, 2025

@Yohahaha CI has passed, could you please take another look?

Yohahaha
Yohahaha approved these changes Apr 16, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@Yohahaha Yohahaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@wForget wForget merged commit 79db240 into apache:main Apr 16, 2025
49 checks passed
@wForget
Copy link
Copy Markdown
Member Author

wForget commented Apr 16, 2025

@Yohahaha Thank you for your review, merged to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Yohahaha Yohahaha Yohahaha approved these changes

Assignees

No one assigned

Labels

CORE works for Gluten Core VELOX

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[VL][1.2] Type mismatch error in VectorHasher

2 participants

@wForget @Yohahaha