REGR: Timestamp - series is failing

[jreback]

I think this changed when the subtraction in Timestamp was changed to be minus adding.
#6544

In [1]: s = pd.Series(pd.date_range('2012-1-1', periods=3, freq='D'))

In [2]: m = s.max()

In [3]: x = m.to_datetime()

In [4]: x - s
Out[4]: 
0   2 days
1   1 days
2   0 days
dtype: timedelta64[ns]

In [5]: m - s

TypeError: ufunc 'negative' did not contain a loop with signature matching types dtype('<M8[ns]') dtype('<M8[ns]')

[jreback]

cc @rosnfeld

can you take a look ?

I thought their were tests for this, but I think maybe only in the docs.

I think need to allow Series on the rhs to work as well (and ad some tests)

[rosnfeld]

Sure, I can take a look at this, though I am on the road and won't have a dev set-up until Monday.

Out of curiosity - how was this caught?

[jreback]

np

was building the docs and it errored
in end of timeseries.rst

[rosnfeld]

Ok, I think this probably has something to do with the strange nature of datetime arithmetic due to lack of an absolute timescale. One can subtract datetimes from one another to get a timedelta, but one can't add them or negate them. The code sees "x - y" and "x + (-y)" quite differently.

I'll take a look at it.

[rosnfeld]

I would note that in 0.13.1, no other standard date/time object supports arithmetic with Timestamp in the right-hand-side.
i.e. while datetime.datetime - Timestamp works, none of datetime.timedelta +/- Timestamp, np.datetime64 - Timestamp, np.timedelta64 +/- Timestamp work.

(just for reference, maybe we'd want to change this someday)

[jreback]

I wouldn't except any of those 2 work (nor is it easy to change).

datetime/Timestamp - Series is the only really import one (for symmetry with Series - datetime/Timestamp).

similary
td +/- Series also works (these work because they end of calling the radd/rsub methods of Series). That's the only reason.

The change to make datetime sub work right bypasses this, that's the problem!

so I think Timestamp.__sub__ needs to use handle Series (as in calling the datetime.__sub__(a_series)) which is was was happengin

[rosnfeld]

I'm just trying to figure out the lay of the land so that we have the right tests in place and I don't break anything else.

Another thing I noted is that if the dt/ts has timezone info, then (as of 0.13.1) the Series doesn't work on the RHS:

In [1]: dr = pd.date_range('2012-01-01', periods=3, freq='D', tz='US/Eastern')

In [2]: dt = datetime.datetime(2014, 3, 4, tzinfo=dr.tzinfo)

In [3]: dt - pd.Series(dr)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-80753e798f9a> in <module>()
----> 1 dt - pd.Series(dr)

/home/andrew/venv/pandas_shootout_v0.13.1/local/lib/python2.7/site-packages/pandas/core/ops.pyc in wrapper(left, right, name)
    462         if isinstance(right, pd.DataFrame):
    463             return NotImplemented
--> 464         time_converted = _TimeOp.maybe_convert_for_time_op(left, right, name)
    465 
    466         if time_converted is None:

/home/andrew/venv/pandas_shootout_v0.13.1/local/lib/python2.7/site-packages/pandas/core/ops.pyc in maybe_convert_for_time_op(cls, left, right, name)
    429         if name.startswith('__r'):
    430             name = "__" + name[3:]
--> 431         return cls(left, right, name)
    432 
    433 

/home/andrew/venv/pandas_shootout_v0.13.1/local/lib/python2.7/site-packages/pandas/core/ops.pyc in __init__(self, left, right, name)
    245 
    246         lvalues = self._convert_to_array(left, name=name)
--> 247         rvalues = self._convert_to_array(right, name=name, other=lvalues)
    248 
    249         self.is_timedelta_lhs = com.is_timedelta64_dtype(left)

/home/andrew/venv/pandas_shootout_v0.13.1/local/lib/python2.7/site-packages/pandas/core/ops.pyc in _convert_to_array(self, values, name, other)
    327             elif not (isinstance(values, (pa.Array, pd.Series)) and
    328                       com.is_datetime64_dtype(values)):
--> 329                 values = tslib.array_to_datetime(values)
    330             elif isinstance(values, pd.DatetimeIndex):
    331                 values = values.to_series()

/home/andrew/venv/pandas_shootout_v0.13.1/local/lib/python2.7/site-packages/pandas/tslib.so in pandas.tslib.array_to_datetime (pandas/tslib.c:16614)()

ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True

[jreback]

works in master....always check master!

I fixed this type of thing recently.

[rosnfeld]

Good catch, you're right. I was trying to check the behavior prior to any of my changes and 0.13.1 was pretty recent but... not precise enough.