[C++] Add support for %Z to strptime

[asfimport]

While lubridate does not support %Z flag for strptime Arrow could.

Changes to C++ kernels might be required for support on all platforms, but that shouldn't block implementation as kStrptimeSupportsZone flag can be used, see proposal.

Reporter: Rok Mihevc / @rok
Watchers: Rok Mihevc / @rok

Related issues:

• [C++] Strptime issues umbrella (is a child of)

_{Note: This issue was originally created as ARROW-17398. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
This is labeled as an R issue, but it is essentially a feature request for the C++ kernel, right? Because as far as I know, currently it "recognizes" the %Z flag (so you can parse strings that have a timezone name in it), but then simply ignores any matching name in the string, based on what I wrote in ARROW-15884

[asfimport]

Rok Mihevc / @rok:
Right! Shall we label it as such and keep a note that the R change is needed as well?

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Yes, changed the label in the PR title.

On the actual topic: do you know to what extent the strptime implementations we use actually support this?

In the Python docs, see note number 6) in https://docs.python.org/3/library/datetime.html#technical-detail about the "%Z" support, and for example there they say that Python's implementation only supports your local time zone (depending on your computer's locale) + "UTC" / "GMT"

[asfimport]

Rok Mihevc / @rok:
We use musl strptime on windows and regular strptime elsewhere (source). My assumption is that all timezone information is currently ignored, simple test (on M1):

import pyarrow.compute as pc
import pyarrow as pa
pc.strptime(pa.array(["2022-01-01 12:00 CET"]), format="%Y-%m-%d %H:%M %Z", unit="ms")

Returns 2022-01-01 12:00:00.000 with type TimestampType(timestamp[ms])
While I would expect 2022-01-01 11:00:00.000 with type TimestampType(timestamp[ms], "UTC)

date.h we vendor supports timezone parsing but is significantly slower as per this comment.

I'm not sure how to continue but I expect there is some significant work to be done here.

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:

My assumption is that all timezone information is currently ignored

Indeed, see also the examples in the issue I mentioned above: ARROW-15884

date.h we vendor supports timezone parsing

That's not directly what I understand from that link. It seems to indicate there is a way to support parsing something like "1999-10-31 00:30:00 US/Pacific PST", but for this issue I think we actually want to parse "1999-10-31 00:30:00 PST"?

There might also not be an actual solution for this, without requiring more user input (in general, those time zone abbreviations like PST, EST, CET, etc are not fully unambiguous)

[asfimport]

Rok Mihevc / @rok:

That's not directly what I understand from that link. It seems to indicate there is a way to support parsing something like "1999-10-31 00:30:00 US/Pacific PST", but for this issue I think we actually want to parse "1999-10-31 00:30:00 PST"?

You're right. But date.h docs do state that timezone names and abbreviations can be parsed:

%Z	The time zone abbreviation or name. A single word is parsed. This word can only contain characters from the basic source character set ([lex.charset] in the C++ standard) that are alphanumeric, or one of '_', '/', '-' or '+'.

There might also not be an actual solution for this, without requiring more user input (in general, those time zone abbreviations like PST, EST, CET, etc are not fully unambiguous)

Are you referring to DST jumps or "name overloading"? For completeness: list of timezones including abbreviations.

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
I think I am referring to name overloading (although I am mostly just repeating what I hear others say ;)). For example CST can be both China Standard Time as Central Standard Time.

You're right. But date.h docs do state that timezone names and abbreviations can be parsed:

Interesting (though in the source code I don't directly find support for that), so we will have to test in practice to see how much it supports it.

[asfimport]

Rok Mihevc / @rok:

I think I am referring to name overloading (although I am mostly just repeating what I hear others say ). For example CST can be both China Standard Time as Central Standard Time.

CST is a good example. I suppose we'll have to offer a resolution mechanism or error.

Interesting (though in the source code I don't directly find support for that), so we will have to test in practice to see how much it supports it.

Interesting inedeed.