[C++][Python] Refactor python to arrow conversions based on a reusable conversion API #26017
Description
We have a lot of technical debt accumulated in the python to arrow conversion code paths including hidden bugs. We need to simplify the implementation.
Reporter: Krisztian Szucs / @kszucs
Assignee: Krisztian Szucs / @kszucs
Related issues:
- [Python] ListArray has trouble with sizes greater than kMaximumCapacity (fixes)
- [Python] Tzinfo - string roundtrip fails on pytz.StaticTzInfo objects (fixes)
- [C++][Python] Auto chunking nested array containing binary-like fields result malformed output (fixes)
- [C++] Dictionary is unset when calling DictionaryArray.GetScalar for null values (fixes)
- [Python] ArrowCapacityError when doing Table.from_pandas with large dataframe (fixes)
- [Python] Support constructing dictionary array directly through pa.array() (fixes)
- [C++][Python] Support constructing StructArray from list of key-value pairs (fixes)
- [Python] Produce chunked arrays for nested types in pyarrow.array (fixes)
- [Python] Add custom pickle reducers for DictionaryScalar (fixes)
- [Python] StructScalar.as_py() fails if the type has duplicate field names (is related to)
- [C++][Python] Do not collect python iterators if not necessary (is related to)
- [Python] Consider to raise or normalize if a timezone aware datetime.time object is encountered during conversion (is related to)
- [C++] Add an Append method to the time builders which validates the input range (is related to)
PRs and other links:
Note: This issue was originally created as ARROW-9992. Please see the migration documentation for further details.