-
Notifications
You must be signed in to change notification settings - Fork 3k
Docs: Add the description of creating a table using DataFrameWriterV2 with a table location #6729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
szehon-ho
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great to document, I left a comment suggestion below.
docs/spark-writes.md
Outdated
| .createOrReplace() | ||
| ``` | ||
|
|
||
| You can specify an Iceberg table location such as the `LOCATION` clause in SQL by add the `location` paramter to the `tableProperty`: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: I know there's a bad example above, but typically we avoid 'you' or 'me' in documentation. you can see other reviews, for example: #4301 (comment)
Also, I'm not sure why we reference "LOCATION" clause in SQL, which is documented elsewhere.
How about:
The Iceberg table location can also be specified by the `location` table configuration
Let me know what you think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for reviewing this and the great suggenstion, Szehon! The following your suggestion is simpler and makes sense. Based on your suggestion, I updated a bit; ... table configuration to table property. How about this?
The Iceberg table location can also be specified by the `location` table property.
For your comment;
Nit: I know there's a bad example above, but typically we avoid 'you' or 'me' in documentation. you can see other reviews, for example: #4301 (comment)
Thanks for pointing out, I understand it.
Also, I'm not sure why we reference "LOCATION" clause in SQL, which is documented elsewhere.
I tried saying the tableProperty("location", "...") is the same as LOCATION in SQL, but the suggestion is better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes that will work. Thanks!
|
Thanks for reviewing! I updated the line based on the discussion. It would be great if you review it. |
|
Merged, thanks @tomtongue ! |
This change adds how to run CTAS using DataFrame V2 API with specifying a table location. As a background, some iceberg users ask how to create an iceberg table by DataFrame V2 with specifying a table location, and it's not easy to find the information about the parameter.