Schema issue between Arrow and PyIceberg

[asheeshgarg]

Apache Iceberg version

1.4.1 (latest release)

Query engine

Other

Please describe the bug 🐞

@Fokko we have a table in iceberg which has some of the column names begin with numbers. We are able to scan the table using PyIceberg. When try to bind it to Arrow or DuckDB we see its Arrow invalid FieldRef.Name no match for field.

What we observe in in Arrow the field name beginning with number like 2030_ABC is renamed to _2030_ABC while the schema on iceberg is correct to define it as 2030_ABC which is in original data. Which trigger this issue.

Seem more of Arrow Bug happy to open it at Arrow project. Let me know

[Fokko]

Thanks @asheeshgarg for raising this. The PyIceberg repository has been moved to https://github.com/apache/iceberg-python. To get the quickest answers, it is best to raise the question over there.

We recently merged a PR that fixes this when reading Avro fields: apache/iceberg-python#83

Can you try if this issue still persists on the main branch? You can easily install it using:

pip install "git+https://github.com/apache/iceberg-python.git#egg=pyiceberg[arrow]"

If you run into anything, could you also share the error? Thanks 🙌

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'