HIVE-28059: Iceberg REST Catalog #5145
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- add CatalogClass conf to instantiate a HiveCatalog (thrift client); - added metrics;
- add CatalogClass conf to instantiate a HiveCatalog (thrift client); - added metrics;
- Introduce HiveActor abstract class that captures all methods needed by the (Iceberg) catalog to interact with Hive (HMS); - Implement thrift-client (HiveCatalogActor) and embedded client (HMSActor); - Removed property specifying catalog implementation; - Added property specifying HiveActor implementation class (hive.iceberg.catalog.actor.class); - Refactored tests accordingly;
added 2 commits
March 20, 2024 23:29
- Introduce HiveActor abstract class that captures all methods needed by the (Iceberg) catalog to interact with Hive (HMS); - Implement thrift-client (HiveCatalogActor) and embedded client (HMSActor); - Removed property specifying catalog implementation; - Added property specifying HiveActor implementation class (hive.iceberg.catalog.actor.class); - Refactored tests accordingly;
- Introduce HiveActor abstract class that captures all methods needed by the (Iceberg) catalog to interact with Hive (HMS); - Implement thrift-client (HiveCatalogActor) and embedded client (HMSActor); - Removed property specifying catalog implementation; - Added property specifying HiveActor implementation class (hive.iceberg.catalog.actor.class); - Refactored tests accordingly;
zratkai
reviewed
Dec 12, 2024
| "hive.metastore.catalog.servlet.path", "icecli", | ||
| "Catalog servlet path component of URL endpoint." | ||
| ), | ||
| CATALOG_SERVLET_PORT("hive.metastore.catalog.servlet.port", |
Contributor
There was a problem hiding this comment.
Please use more appropriate name here. I could not find out neither from the name, nor from the description this property is for Iceberg Rest Server.
zratkai
reviewed
Dec 12, 2024
| "hive.metastore.properties.namespaces", "", | ||
| "Property-maps namespace map; a JSON map where keys are namespace names, values namespace" + | ||
| "class and/or static factory method"), | ||
| CATALOG_SERVLET_PATH("hive.metastore.catalog.servlet.path", |
Contributor
There was a problem hiding this comment.
Please use more appropriate name here. I could not find out neither from the name, nor from the description this property is for Iceberg Rest Server
zratkai
reviewed
Dec 12, 2024
| " 0 will let the system determine the catalog server port," + | ||
| " positive value will be used as-is." | ||
| ), | ||
| CATALOG_SERVLET_AUTH("hive.metastore.catalog.servlet.auth", |
Contributor
There was a problem hiding this comment.
Please use more appropriate name here. I could not find out neither from the name, nor from the description this property is for Iceberg Rest Server
zratkai
reviewed
Dec 12, 2024
| import org.slf4j.LoggerFactory; | ||
|
|
||
|
|
||
| public class HMSCatalogServer { |
Contributor
There was a problem hiding this comment.
I think HMSRESTCatalogServer would be better.
zratkai
reviewed
Dec 12, 2024
| * The RESTCatalogServlet provides a servlet implementation used in combination with a | ||
| * RESTCatalogAdaptor to proxy the REST Spec to any Catalog implementation. | ||
| */ | ||
| public class HMSCatalogServlet extends HttpServlet { |
Contributor
There was a problem hiding this comment.
I think HMSRESTCatalogServlet would be better.
zratkai
reviewed
Dec 12, 2024
| * Original @ https://github.com/apache/iceberg/blob/main/core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java | ||
| * Adaptor class to translate REST requests into {@link Catalog} API calls. | ||
| */ | ||
| public class HMSCatalogAdapter implements RESTClient { |
Contributor
There was a problem hiding this comment.
I think HMSRESTCatalogAdapter would be better.
zratkai
reviewed
Dec 12, 2024
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| public class HMSCatalogActor implements HiveActor { |
Contributor
There was a problem hiding this comment.
I think HMSRESTCatalogActor would be better.
zratkai
suggested changes
Dec 12, 2024
Contributor
zratkai
left a comment
zratkai
left a comment
There was a problem hiding this comment.
Please check my comments!
Contributor
Author
Will do :-) |
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This is a basic implementation of the Iceberg REST Catalog as a service embedded within HMS.
Why are the changes needed?
For Hive users that have started using Iceberg to store data and want to expose those same tables to external query engines, implementing the Iceberg REST catalog embedded within HMS allows an easy upgrade and deployment path.
Although a dedicated service to host the REST catalog is certainly desirable, it also implies deploying and managing another service.
Does this PR introduce any user-facing change?
It does since HMS then 'hosts' a servlet that implements the Iceberg REST Catalog API.
Is the change a dependency upgrade?
No.
How was this patch tested?
Unit tests.