[Python][CI] Build with pandas master/nightly failure related to timedelta64 resolution #33286
Description
The nightly python builds using the pandas development version are failing: https://github.com/ursacomputing/crossbow/actions/runs/3269767207/jobs/5377649455
Example failure:
____________________ test_parquet_2_0_roundtrip[None-True] _____________________
tempdir = PosixPath('/tmp/pytest-of-root/pytest-0/test_parquet_2_0_roundtrip_Non0')
chunk_size = None, use_legacy_dataset = True
@pytest.mark.pandas
@parametrize_legacy_dataset
@pytest.mark.parametrize('chunk_size', [None, 1000])
def test_parquet_2_0_roundtrip(tempdir, chunk_size, use_legacy_dataset):
df = alltypes_sample(size=10000, categorical=True)
filename = tempdir / 'pandas_roundtrip.parquet'
arrow_table = pa.Table.from_pandas(df)
assert arrow_table.schema.pandas_metadata is not None
_write_table(arrow_table, filename, version='2.6',
coerce_timestamps='ms', chunk_size=chunk_size)
table_read = pq.read_pandas(
filename, use_legacy_dataset=use_legacy_dataset)
assert table_read.schema.pandas_metadata is not None
read_metadata = table_read.schema.metadata
assert arrow_table.schema.metadata == read_metadata
df_read = table_read.to_pandas()
> tm.assert_frame_equal(df, df_read)
E AssertionError: Attributes of DataFrame.iloc[:, 12] (column name="timedelta") are different
E
E Attribute "dtype" are different
E [left]: timedelta64[s]
E [right]: timedelta64[ns]
opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/tests/parquet/test_data_types.py:76: AssertionErrorReporter: Joris Van den Bossche / @jorisvandenbossche
Assignee: Joris Van den Bossche / @jorisvandenbossche
Related issues:
PRs and other links:
Note: This issue was originally created as ARROW-18088. Please see the migration documentation for further details.