Skip to content

Memory pointer from external source (e.g, FFI) is not aligned for empty list[str] #7549

@ion-elgreco

Description

@ion-elgreco

Describe the bug
Can't consume arrow data with empty list[str] over FFI

To Reproduce

import polars as pl
from arro3.core import Table
df = pl.DataFrame(
    {
        "bar": [[]],
    }, schema={"bar": pl.List(pl.String)}
)

Table(df)

Expected behavior
Pyarrow seems to be able to consume this empty list[str] fine,

import polars as pl
import pyarrow as pa
df = pl.DataFrame(
    {
        "bar": [[]],
    }, schema={"bar": pl.List(pl.String)}
)

pa.table(df)

Additional context
Created also issue in polars repo: pola-rs/polars#22935, since it's not clear to me where the issue lies, wether it's polars buffers being misaligned or arrow-rs incorrectly identifying it because pyarrow seems to be able to consume it fine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions