Skip to content

[JS] Crash when reading IPC file with Union types #37133

@kylebarron

Description

@kylebarron

Describe the bug, including details regarding any error messages, version, and platform.

When loading an IPC file in Arrow JS using arrow.tableFromIPC, I get a crash when the file contains either a Sparse Union or a Dense Union column.

TypeError: Cannot destructure property 'length' of '(intermediate value)(intermediate value)(intermediate value)' as it is undefined.
 ❯ VectorLoader.readData node_modules/apache-arrow/src/visitor/vectorloader.ts:125:56
 ❯ VectorLoader.visitBool node_modules/apache-arrow/src/visitor/vectorloader.ts:61:113
 ❯ VectorLoader.visit node_modules/apache-arrow/src/visitor.ts:26:48
 ❯ VectorLoader.visit node_modules/apache-arrow/src/visitor/vectorloader.ts:54:22
 ❯ node_modules/apache-arrow/src/visitor.ts:23:44
 ❯ VectorLoader.visitMany node_modules/apache-arrow/src/visitor.ts:23:22
 ❯ VectorLoader.visitSparseUnion node_modules/apache-arrow/src/visitor/vectorloader.ts:103:150
 ❯ VectorLoader.visit node_modules/apache-arrow/src/visitor.ts:26:48
 ❯ VectorLoader.visit node_modules/apache-arrow/src/visitor/vectorloader.ts:54:22

Repro:

git clone https://github.com/kylebarron/arrow-js-ffi
git checkout 50dd5d1
yarn install
yarn test

In particular, this array is generated with this snippet from Python:

def sparse_union_array() -> pa.Array:
    """Generate a sparse union array

    This is derived from the example here https://arrow.apache.org/docs/python/data#union-arrays
    """
    # First child array
    xs = pa.array([5, 6, 7])

    # Second child array
    ys = pa.array([False, False, True])

    # Type mapping
    types = pa.array([0, 1, 1], type=pa.int8())

    # Union array
    union_arr = pa.UnionArray.from_sparse(types, [xs, ys])

    assert isinstance(union_arr, pa.UnionArray)
    assert union_arr[0].as_py() == 5
    assert union_arr[1].as_py() is False
    assert union_arr[2].as_py() is True

    return union_arr

which is then dumped into an uncompressed IPC file, which can't be read in JS.

Component(s)

JavaScript

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions