DatetimeIndex comparison with ndim>1 incorrect or raises

[jbrockmendel]

Discovered while testing #18162.

dti = pd.date_range('2016-01-01', periods=3, tz='US/Pacific')
df = pd.Series(dti).to_frame()
arr = np.array([dti.tolist()]).T  # <-- object dtype, correctly retains TZ
arr2 = np.array([dti]).T          # <-- datetime64[ns], drops TZ

>>> dti == df
[...]
  File "pandas/core/indexes/datetimes.py", line 145, in _ensure_datetime64
    raise TypeError('%s type object %s' % (type(other), str(other)))

>>> df == dti
  File "<stdin>", line 1, in <module>
  File "pandas/core/ops.py", line 1302, in f
    try_cast=False)
  File "pandas/core/frame.py", line 4012, in _combine_const
[...]
  File "pandas/core/internals.py", line 121, in __init__
    '{mgr}'.format(val=len(self.values), mgr=len(self.mgr_locs)))
ValueError: Wrong number of items passed 3, placement implies 1

>>> dti == arr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/core/indexes/datetimes.py", line 126, in wrapper
    o_mask = other.view('i8') == libts.iNaT
  File "/usr/local/lib/python2.7/site-packages/numpy/core/_internal.py", line 377, in _view_is_safe
    raise TypeError("Cannot change data-type for object array.")
TypeError: Cannot change data-type for object array.

>>> dti == arr2    # <-- this one is fixed by #18162
array([[ True, False, False],
       [False,  True, False],
       [False, False,  True]], dtype=bool)

>>> df == arr   # <-- this one is OK
      0
0  True
1  True
2  True

>>> df == arr2  # <-- should raise TypeError
       0
0  False
1  False
2  False

>>> df[0] == arr2[:, 0]  # <-- should raise TypeError
0    True
1    True
2    True

[mroeschke]

This looks correct now @jbrockmendel?

In [60]: dti == df
ValueError: Unable to coerce to Series, length must be 1: given 3

In [61]: df == dti
ValueError: Unable to coerce to Series, length must be 1: given 3

In [62]: dti == arr
ValueError: Buffer has wrong number of dimensions (expected 1, got 2)

In [63]: df == arr2
TypeError: Cannot compare tz-naive and tz-aware datetime-like objects

df[0] == arr2[:, 0]
TypeError: Cannot compare tz-naive and tz-aware datetime-like objects

[jbrockmendel]

I’d prefer if 62 broadcast, but that’s not something we do with any consistency. The rest are clearly much improved.

[mroeschke]

I think these all look correct now that we store Timestamp objects in numpy arrays. Could use tests

In [19]: dti == df
ValueError: Unable to coerce to Series, length must be 1: given 3

In [20]: df == dti
ValueError: Unable to coerce to Series, length must be 1: given 3

In [21]: dti == arr
Out[21]:
array([[ True, False, False],
       [False,  True, False],
       [False, False,  True]])

In [22]: df == arr2
Out[22]:
      0
0  True
1  True
2  True

In [23]: df[0] == arr2[:, 0]
Out[23]:
0    True
1    True
2    True
Name: 0, dtype: bool

[ashishthanki]

Hey @mroeschke have the tests for this been written? Can I work on it as my first time issue?

[jbrockmendel]

Can I work on it as my first time issue?

Yes.

have the tests for this been written?

Hard to tell. I'd suggest adding a check that intentionally raises in the relevant cases and see if anything in the test suite breaks.