[Java/Python] Support VarCharVector / StringArray in pyarrow.Array.from_jvm

[asfimport]

Follow-up after apache/arrow-java#420: Currently only primitive arrays are supported in pyarrow.Array.from_jvm as it uses pyarrow.Array.from_buffers underneath. We should extend one of the two functions to be able to deal with string arrays. There is a currently failing unit test test_jvm_string_array in pyarrow/tests/test_jvm.py to verify the implementation.

Reporter: Uwe Korn / @xhochy

Related issues:

• [Java/Python] in-process vector sharing from Java to Python (is a child of)
• [Python] Support binary, utf8, and nested types in Array.from_buffers (depends upon)

_{Note: This issue was originally created as ARROW-2607. Please see the migration documentation for further details.}

[asfimport]

Tim Swast / @tswast:
I'm very interested in this issue, as it would be extremely useful for parsing files into Arrow tables from numba. I would expect to be able to do the following:

my_string_array = pyarrow.Array.from_buffers(
  pyarrow.string(),
  row_count,
  [
    pyarrow.py_buffer(my_string_nullmask),
    pyarrow.py_buffer(my_string_offsets),
    pyarrow.py_buffer(my_string_bytes),
  ],
)

But I get :

File "pyarrow/array.pxi", line 578, in pyarrow.lib.Array.from_buffers
NotImplementedError: from_buffers is only supported for primitive arrays yet.
I suppose if I wanted to contribute this fix, I should start looking at pyarrow/array.pxi first?

[asfimport]

Wes McKinney / @wesm:
Yes, indeed. For string/utf8 I think it should already work if you remove the exception and add some unit tests. Nested data will require a bit more work. I suggest also calling "ValidateArray" on the resulting array to guard against malformed output

[asfimport]

Wes McKinney / @wesm:
I just opened ARROW-5531 about addressing the Array.from_buffers shortcoming in particular

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.