[Feature] Generate Right and Completed Iceberg Metadata when using a centralized and unified Hive Metastore like AWS Glue Data Catalog

[Moonlight-CL]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Now Paimon Can't generate right and completed Iceberg metadatas When using a centralized and unified Hive Metastore like AWS Glue Data Catalog.

When generate iceberg metadata for a paimon table, first it will check the database and table exist or not, if we use a centralized and unified Hive metastore, the check will return true, and then paimon will not generate a whole new iceberg metadata, and just update the table properties like metadata_location. All this make spark, flink and other engines can not identity it as a Iceberg table.

Solution

1. Add metadata.iceberg.database and metadata.iceberg.table options on paimon table when generating Iceberg metadata.
2. If user set the options, we will takes priority over the database / table derived from the FileStoreTable

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!