-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[feature](hudi) Step2: Support query hudi external table(include cow and mor table) #9752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…as string after scan in be.
fe/fe-core/src/main/java/org/apache/doris/planner/HudiScanNode.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/planner/HudiScanNode.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/planner/HudiScanNode.java
Outdated
Show resolved
Hide resolved
morningman
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
@morningman @Jibing-Li please help to review |
morningman
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
Proposed changes
Issue Number: #9557
Support query hudi external table in Doris.
This pr support query cow and mor hudi table.
When hudi table is a mor table, only support read optimized query mode.
This is the second pr to support hudi external table.
Problem Summary:
Describe the overview of changes.
The propose of the pr is:
support query cow and mor hudi table.
Design
Create a HudiScanNode to generate scan range parameters.
HudiScanNode use HoodieParquetInputFormat to get all scan splits and assemble brokerRangeDesc
we use broker_scan_node to scan parquet files that send by fe.
query cow table
query mor table
query cow partition table
query the hudi table with some column information specified
query two columns from hudi table
Checklist(Required)
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...