[Python] Add timezone information when printing TimestampArray

[AlenkaF]

Describe the enhancement requested

PrettyPrint for Timestamp is currently printing values in UTC even when the timezone is defined. This can get confusing and there is a PR open with a simple fix of adding "Z" to the end of the string in case timezone is defined:
#39272

This way we can at least see which tz is the data printed in.

It would also be good to add timezone information when printing the array, for example:

# Adding timezone information in new line
>>> pa.array([0], pa.timestamp('s', tz='+02:00'))
<pyarrow.lib.TimestampArray object at 0x125319de0>
<timestamp[s, tz=+02:00]>
[
  1970-01-01 00:00:00Z
]

# Or at the end of first line (not very clear in my opinion)
>>> pa.array([0], pa.timestamp('s'))
<pyarrow.lib.TimestampArray object at 0x125319e40><timestamp[s]> 
[
  1970-01-01 00:00:00
]

In Python this can be done by adding a separate __repr__ to TimestampArray class.

Would something similar also be needed for R or is timezone information available when printing an Array? cc @paleolimbot

Component(s)

Python, R

[paleolimbot]

It looks like we already print the timezone?

arrow::as_arrow_array(Sys.time())
#> Array
#> <timestamp[us, tz=America/Halifax]>
#> [
#>   2023-12-20 13:33:42.632165
#> ]

...and it also looks like our abbreviated printer displays it too:

dplyr::glimpse(arrow::arrow_table(ts = Sys.time()))
#> Table
#> 1 rows x 1 columns
#> $ ts <timestamp[us, tz=America/Halifax]> 2023-12-20 09:34:44

The Z suffix is definitely a good idea though!

[AlenkaF]

Thanks for the info Dewey!

[jorisvandenbossche]

While we are at improving the repr of arrays, we can probably leave out the boilerplate in <pyarrow.lib.TimestampArray object at 0x125319de0> (that's just python's default repr), and limit it to something like <pyarrow.TimestampArray> in general (similarly for other array types).
And then that also makes it easier for TimestampArray to add additional information. Something like <pyarrow.TimestampArray[us, tz=America/Halifax]> could then be an option (although I am not fully sure I like adding a [] parametrization to the array, because that is typically only something we do for types, not arrays)

[jorisvandenbossche]

Some variants:

<pyarrow.TimestampArray[us, tz=America/Halifax]>
<pyarrow.TimestampArray: timestamp[us, tz=America/Halifax]>
<pyarrow.TimestampArray<timestamp[us, tz=America/Halifax]>>
<pyarrow.TimestampArray><timestamp[us, tz=America/Halifax]>

(maybe the first is actually fine, and also the most succinct)

[ianmcook]

I think this and other improvements to the way timezone-aware timestamps are printed would be very helpful for users.

The way PyArrow currently prints timezone-aware timestamp values can be very confusing. For example, you might try to create a Table like this:

from datetime import datetime
import pyarrow as pa

t = pa.table(
    {'ts': [datetime(1969, 1, 1, 1, 1, 1)]},
    schema=pa.schema([("ts", pa.timestamp("us", tz="America/New_York"))])
)

When you print it, it looks like the time represents the 01:01:01 EST:

t
## pyarrow.Table
## ts: timestamp[us, tz=America/New_York]
## ----
## ts: [[1969-01-01 01:01:01.000000Z]]

But upon closer inspection, it is actually representing the time 01:01:01 UTC which converts to 20:01:01 EST:

t["ts"][0]
## <pyarrow.TimestampScalar: '1968-12-31T20:01:01.000000-0500'>

[rok]

We could display local time (using local_timestamp kernel), but then we should make it clear we're displaying wall time.