-
Notifications
You must be signed in to change notification settings - Fork 4k
Description
We currently do not read nor write the version 2 logical types. This is mainly because we do not have a mapping for it from parquet-format-rs.
To implement this, we can:
-
convert "parquet::basic::LogicalType" to "parquet::basic::ConvertedType"
-
implement "parquet::basic::LogicalType" which mirrors "parquet_format::LogicalType"
-
create a mapping between ConvertedType and LogicalType
-
write LogicalType to "parquet_format::SchemaElement" if v2 of the writer is used
This would be a good starting point for implementing 2.6 types (UUID, NANOS precision time & timestamp).
Follow-up work would be: -
parsing v2 of the schema [ARROW-11365]
-
Using v2 in the Arrow writer (mostly schema conversion)
-
Supporting nanosecond precision time & timestamp
Reporter: Neville Dipale / @nevi-me
Assignee: Neville Dipale / @nevi-me
PRs and other links:
Note: This issue was originally created as ARROW-11803. Please see the migration documentation for further details.