Is your feature request related to a problem or challenge? Please describe what you are trying to do.
|
/// TODO: How would a caller request a struct or list type where the fields/elements can be any |
|
/// variant? Caller can pass None as the requested type to fetch a specific path, but it would |
|
/// quickly become annoying (and inefficient) to call `variant_get` for each leaf value in a struct or |
|
/// list and then try to assemble the results. |
Given data like:
{"user": {"id": 123, "metadata": {"tags": ["a"], "score": 95.5}}}
Users want to extract user with:
- id as Int32
- metadata as Variant (not expanded)
Currently, this requires multiple variant_get calls and manual StructArray assembly.
Describe the solution you'd like
Use ExtensionType to mark fields that should remain as Variant:
let field = Field::new("user", DataType::Struct(Fields::from(vec![
Field::new("id", DataType::Int32, true),
Field::new("metadata", ..., true).with_extension_type(VariantType), // Keep as Variant
])), true);
let result = variant_get(&array, GetOptions::new().with_as_type(Some(Arc::new(field))))?;
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
arrow-rs/parquet-variant-compute/src/variant_get.rs
Lines 273 to 276 in 5ba4515
Given data like:
Users want to extract user with:
Currently, this requires multiple variant_get calls and manual StructArray assembly.
Describe the solution you'd like
Use ExtensionType to mark fields that should remain as Variant: