[Python][CI] Failing test_dateutil_tzinfo_to_string due to new release of python-dateutil

[AlenkaF]

Describe the bug, including details regarding any error messages, version, and platform.

test_dateutil_tzinfo_to_string started failing with:

         tz = dateutil.tz.gettz('Europe/Paris')
>       assert pa.lib.tzinfo_to_string(tz) == 'Europe/Paris'
E       AssertionError: assert 'Europe/Monaco' == 'Europe/Paris'
E         - Europe/Paris
E         + Europe/Monaco

see: https://github.com/ursacomputing/crossbow/actions/runs/8103088008/job/22164107523#step:6:4305.

This is most probably due to new release of python-dateutil package.

The test is failing on the CI now also, with dateutil 2.9.0: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/49322144?fullLog=true

Upstream issue: dateutil/dateutil#1353

Component(s)

Continuous Integration, Python

[AlenkaF]

cc @raulcd

[jorisvandenbossche]

I can't reproduce this on Linux, so most probably a Windows issue (or related to the exact tzdata being used there, since for me locally it's using the system provided data)

Might be good to report upstream?

[jorisvandenbossche]

The logic we use to convert a dateutil tz to a string:

arrow/python/pyarrow/src/arrow/python/datetime.cc

Lines 554 to 568 in 2b194ad

	// if dateutil is installed and tzinfo is an instance of dateutil.tz.tzfile
	if (module_dateutil.obj() != nullptr &&
	PyObject_IsInstance(tzinfo, class_tzfile.obj())) {
	OwnedRef _filename(PyObject_GetAttrString(tzinfo, "_filename"));
	RETURN_IF_PYERROR();
	std::string result;
	RETURN_NOT_OK(internal::PyUnicode_AsStdString(_filename.obj(), &result));
	// _filename returns a full path in general ('/usr/share/zoneinfo/Europe/Paris')
	// or POSIX name on Windows ('Europe/Paris') - we need a substring in first case
	std::size_t pos = result.find("zoneinfo/");
	if (pos != std::string::npos) {
	return result.substr(pos + 9);
	}
	return result;
	}

(so it might also be that something changed under the hood in dateutil that messes that u. It would be good to verify if the actual tz object in the snippet above, before conversion to a string by pyarrow, also shows "Europe/Monaco")

[AlenkaF]

Agree with you Joris. I would first like to check the behaviour of dateutil, as you mention, but also can't do that on my M1. I could in any case open an issue upstream before, if we think that makes sense.

[raulcd]

The same issue is being reproduced on my PR on the conda feedstock on Windows:
https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=888527&view=logs&j=ab98fee7-5bc6-5c2f-a410-3ab9b2f2e8ca&t=472408ae-fc68-5791-981c-69ea41d2d692&l=38399

    def test_dateutil_tzinfo_to_string():
        pytest.importorskip("dateutil")
        import dateutil.tz
    
        tz = dateutil.tz.UTC
        assert pa.lib.tzinfo_to_string(tz) == 'UTC'
        tz = dateutil.tz.gettz('Europe/Paris')
>       assert pa.lib.tzinfo_to_string(tz) == 'Europe/Paris'
E       AssertionError: assert 'Europe/Monaco' == 'Europe/Paris'
E         - Europe/Paris
E         + Europe/Monaco

Edit: added on Windows

[jorisvandenbossche]

A temporary skip was added in #40486