Skip to content

[C++][Parquet]: Add support for list view and large list view  #38849

@mapleFU

Description

@mapleFU

Describe the enhancement requested

Parquet itself doesn't has a LIST_VIEW type, however, we can read/write a LIST_VIEW inside parquet.

Some notes:

  1. During loading, I think making a arrow::list is ok, and converting a List to ListView is so convenient. So maybe we can first support write
  2. For parquet::arrow::SchemaManifest, maybe it's ok to just store a list as metadata?

TODO:

  1. For writer, we need support PathInfo. The logic is so similiar to origin list node, however, we need support a new kind of RangeSelector. We don't need to support a new list because the writer here doesn't make full use of the contigous array
  2. For schema, we need to handle the parquet/arrow/schema.h well.

Component(s)

C++, Parquet

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions