-
Notifications
You must be signed in to change notification settings - Fork 4k
Closed
Description
Describe the bug, including details regarding any error messages, version, and platform.
When loading an IPC file in Arrow JS using arrow.tableFromIPC, I get a crash when the file contains either a Sparse Union or a Dense Union column.
TypeError: Cannot destructure property 'length' of '(intermediate value)(intermediate value)(intermediate value)' as it is undefined.
❯ VectorLoader.readData node_modules/apache-arrow/src/visitor/vectorloader.ts:125:56
❯ VectorLoader.visitBool node_modules/apache-arrow/src/visitor/vectorloader.ts:61:113
❯ VectorLoader.visit node_modules/apache-arrow/src/visitor.ts:26:48
❯ VectorLoader.visit node_modules/apache-arrow/src/visitor/vectorloader.ts:54:22
❯ node_modules/apache-arrow/src/visitor.ts:23:44
❯ VectorLoader.visitMany node_modules/apache-arrow/src/visitor.ts:23:22
❯ VectorLoader.visitSparseUnion node_modules/apache-arrow/src/visitor/vectorloader.ts:103:150
❯ VectorLoader.visit node_modules/apache-arrow/src/visitor.ts:26:48
❯ VectorLoader.visit node_modules/apache-arrow/src/visitor/vectorloader.ts:54:22
Repro:
git clone https://github.com/kylebarron/arrow-js-ffi
git checkout 50dd5d1
yarn install
yarn test
In particular, this array is generated with this snippet from Python:
def sparse_union_array() -> pa.Array:
"""Generate a sparse union array
This is derived from the example here https://arrow.apache.org/docs/python/data#union-arrays
"""
# First child array
xs = pa.array([5, 6, 7])
# Second child array
ys = pa.array([False, False, True])
# Type mapping
types = pa.array([0, 1, 1], type=pa.int8())
# Union array
union_arr = pa.UnionArray.from_sparse(types, [xs, ys])
assert isinstance(union_arr, pa.UnionArray)
assert union_arr[0].as_py() == 5
assert union_arr[1].as_py() is False
assert union_arr[2].as_py() is True
return union_arrwhich is then dumped into an uncompressed IPC file, which can't be read in JS.
Component(s)
JavaScript