COMPAT: dateutil failing on master

[jreback]

https://travis-ci.org/pandas-dev/pandas/jobs/415820574, this is the numpy-dev build

we are building dateutil from source: dateutil: 2.7.3.dev208+g510359d
is causing the following

=================================== FAILURES ===================================
_____ TestDatetimeIndexTimezones.test_timetz_accessor[dateutil/US/Pacific] _____
[gw1] linux -- Python 3.6.6 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.indexes.datetimes.test_timezones.TestDatetimeIndexTimezones object at 0x7f5d9cb444e0>
tz_naive_fixture = 'dateutil/US/Pacific'
    def test_timetz_accessor(self, tz_naive_fixture):
        # GH21358
        if tz_naive_fixture is not None:
>           tz = dateutil.tz.gettz(tz_naive_fixture)
pandas/tests/indexes/datetimes/test_timezones.py:737: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.6/site-packages/dateutil/tz/tz.py:1546: in __call__
    self.__instances[name] = rv
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:168: in __setitem__
    self.data[key] = KeyedRef(value, self._remove, key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
type = <class 'weakref.KeyedRef'>, ob = None
callback = <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5db13116a8>
key = 'dateutil/US/Pacific'
    def __new__(type, ob, callback, key):
>       self = ref.__new__(type, ob, callback)
E       TypeError: cannot create weak reference to 'NoneType' object
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:335: TypeError
___ TestDatetimeIndexTimezones.test_timetz_accessor[dateutil/Asia/Singapore] ___
[gw1] linux -- Python 3.6.6 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.indexes.datetimes.test_timezones.TestDatetimeIndexTimezones object at 0x7f5d9cb58390>
tz_naive_fixture = 'dateutil/Asia/Singapore'
    def test_timetz_accessor(self, tz_naive_fixture):
        # GH21358
        if tz_naive_fixture is not None:
>           tz = dateutil.tz.gettz(tz_naive_fixture)
pandas/tests/indexes/datetimes/test_timezones.py:737: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.6/site-packages/dateutil/tz/tz.py:1546: in __call__
    self.__instances[name] = rv
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:168: in __setitem__
    self.data[key] = KeyedRef(value, self._remove, key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
type = <class 'weakref.KeyedRef'>, ob = None
callback = <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5db13116a8>
key = 'dateutil/Asia/Singapore'
    def __new__(type, ob, callback, key):
>       self = ref.__new__(type, ob, callback)
E       TypeError: cannot create weak reference to 'NoneType' object
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:335: TypeError
 TestTimestampTZOperations.test_timestamp_timetz_equivalent_with_datetime_tz[dateutil/US/Pacific] 
[gw1] linux -- Python 3.6.6 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.scalar.timestamp.test_timezones.TestTimestampTZOperations object at 0x7f5d9dfd02e8>
tz_naive_fixture = 'dateutil/US/Pacific'
    def test_timestamp_timetz_equivalent_with_datetime_tz(self,
                                                          tz_naive_fixture):
        # GH21358
        if tz_naive_fixture is not None:
>           tz = dateutil.tz.gettz(tz_naive_fixture)
pandas/tests/scalar/timestamp/test_timezones.py:298: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.6/site-packages/dateutil/tz/tz.py:1546: in __call__
    self.__instances[name] = rv
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:168: in __setitem__
    self.data[key] = KeyedRef(value, self._remove, key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
type = <class 'weakref.KeyedRef'>, ob = None
callback = <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5db13116a8>
key = 'dateutil/US/Pacific'
    def __new__(type, ob, callback, key):
>       self = ref.__new__(type, ob, callback)
E       TypeError: cannot create weak reference to 'NoneType' object
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:335: TypeError
 TestTimestampTZOperations.test_timestamp_timetz_equivalent_with_datetime_tz[dateutil/Asia/Singapore] 
[gw1] linux -- Python 3.6.6 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.scalar.timestamp.test_timezones.TestTimestampTZOperations object at 0x7f5d9dfd8668>
tz_naive_fixture = 'dateutil/Asia/Singapore'
    def test_timestamp_timetz_equivalent_with_datetime_tz(self,
                                                          tz_naive_fixture):
        # GH21358
        if tz_naive_fixture is not None:
>           tz = dateutil.tz.gettz(tz_naive_fixture)
pandas/tests/scalar/timestamp/test_timezones.py:298: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.6/site-packages/dateutil/tz/tz.py:1546: in __call__
    self.__instances[name] = rv
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:168: in __setitem__
    self.data[key] = KeyedRef(value, self._remove, key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
type = <class 'weakref.KeyedRef'>, ob = None
callback = <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5db13116a8>
key = 'dateutil/Asia/Singapore'
    def __new__(type, ob, callback, key):
>       self = ref.__new__(type, ob, callback)
E       TypeError: cannot create weak reference to 'NoneType' object
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:335: TypeError
____ TestSeriesDatetimeValues.test_dt_timetz_accessor[dateutil/US/Pacific] _____
[gw1] linux -- Python 3.6.6 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.series.test_datetime_values.TestSeriesDatetimeValues object at 0x7f5d9ebde898>
tz_naive_fixture = 'dateutil/US/Pacific'
    def test_dt_timetz_accessor(self, tz_naive_fixture):
        # GH21358
        if tz_naive_fixture is not None:
>           tz = dateutil.tz.gettz(tz_naive_fixture)
pandas/tests/series/test_datetime_values.py:468: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.6/site-packages/dateutil/tz/tz.py:1546: in __call__
    self.__instances[name] = rv
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:168: in __setitem__
    self.data[key] = KeyedRef(value, self._remove, key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
type = <class 'weakref.KeyedRef'>, ob = None
callback = <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5db13116a8>
key = 'dateutil/US/Pacific'
    def __new__(type, ob, callback, key):
>       self = ref.__new__(type, ob, callback)
E       TypeError: cannot create weak reference to 'NoneType' object
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:335: TypeError
__ TestSeriesDatetimeValues.test_dt_timetz_accessor[dateutil/Asia/Singapore] ___
[gw1] linux -- Python 3.6.6 /home/travis/miniconda3/envs/pandas/bin/python
self = <pandas.tests.series.test_datetime_values.TestSeriesDatetimeValues object at 0x7f5d9d5e5710>
tz_naive_fixture = 'dateutil/Asia/Singapore'
    def test_dt_timetz_accessor(self, tz_naive_fixture):
        # GH21358
        if tz_naive_fixture is not None:
>           tz = dateutil.tz.gettz(tz_naive_fixture)
pandas/tests/series/test_datetime_values.py:468: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/pandas/lib/python3.6/site-packages/dateutil/tz/tz.py:1546: in __call__
    self.__instances[name] = rv
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:168: in __setitem__
    self.data[key] = KeyedRef(value, self._remove, key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
type = <class 'weakref.KeyedRef'>, ob = None
callback = <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5db13116a8>
key = 'dateutil/Asia/Singapore'
    def __new__(type, ob, callback, key):
>       self = ref.__new__(type, ob, callback)
E       TypeError: cannot create weak reference to 'NoneType' object
../../../miniconda3/envs/pandas/lib/python3.6/weakref.py:335: TypeError

[jreback]

cc @pganssle
@jbrockmendel @gfyoung @mroeschke

[pganssle]

When did this start breaking? I haven't merged anything significant in a long while.

[pganssle]

Found the issue on the dateutil side, but I think your test suite may not be doing quite what you want. This discovers a bug that has been on master since April, so I suspect that something has recently changed in your code such that you are no longer getting proper time zones. Unless getting None for those tz.gettz calls is the desired outcome, I think you may need some additional assertions in the test.

Ref:

Issue: dateutil/dateutil#800
PR: dateutil/dateutil#801

[mroeschke]

I agree with @pganssle that test_timetz_accessor isn't grabbing the timezone from the fixture correctly. Instead of using dateutil.tz.gettz to parse all the timezone strings in the fixture, we should be using maybe_get_tz

pandas/pandas/_libs/tslibs/timezones.pyx

Line 86 in cf70d11

cpdef inline object maybe_get_tz(object tz):

[pganssle]

@mroeschke More important than that, these tests should be failing right now, because they don't do what you expect. See here. You should have assert tz is not None in the branch which accesses the dateutil tz.