BUG: binary comparison of numpy.int/float and Series

[dmsul]

This only happens with the numpy object is on the left. It doesn't matter if it's an int or a float. This error does not get raised with DataFrames.

After more poking around, It looks like this actually comres from a change in numpy, between versions 1.8.2 and 1.9.0.

import pandas as pd
import numpy as np

s = pd.Series(np.arange(4))
arr = np.arange(4)

right = s < arr[0]
left = arr[0] > s

Running this yields

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
bug_lhs_numpy.py in <module>()
      6
      7 right = s < arr[0]
----> 8 left = arr[0] > s

C:\Anaconda\lib\site-packages\pandas-0.14.1_236_g989a51b-py2.7-win-amd64.egg\pandas\core\ops.py
ther)
    555             return NotImplemented
    556         elif isinstance(other, (pa.Array, pd.Series, pd.Index)):
--> 557             if len(self) != len(other):
    558                 raise ValueError('Lengths must match to compare')
    559             return self._constructor(na_op(self.values, np.asarray(other)),

TypeError: len() of unsized object

In [2]: type(arr[0])
Out[2]: numpy.int32

[jreback]

left = 0 > s works (e.g. a python scalar). So I think this is being treated as a 0-dim array (its a np.int64) (and not as a scalar when called.) I'll mark as a bug. Feel free to dig in.

[dmsul]

It is indeed being converted at some point to a 0-dim array (ipython %debug). But I'm fairly certain it's something on the numpy side. (As long as numpy<=1.8.2, everything is fine in pandas>=0.14.0. As soon as you bump numpy to 1.9.0, the example code raises the error on every version of pandas.) I think this one is going to be way over my head. But if I happen to learn C before someone else takes care of this, I'll give it a try.

[tvyomkesh]

Thinking about this from a slightly different angle. Series comparison seems to works fine if LHS or RHS is one element list.

In [1]: import pandas as pd
In [2]: import numpy as np
In [3]: [0] < pd.Series(np.arange(4))
Out[3]:
0    False
1     True
2     True
3     True
dtype: bool
In [8]: pd.Series(np.arange(4)) > [0]
Out[8]:
0    False
1     True
2     True
3     True
dtype: bool

Question is should the behavior be modified so that we get the same answer with one element pd.Series or np.ndarray, same as with list or scalar? numpy works with both scalar or one element np.ndarray.

In [9]: np.arange(1) < np.arange(4)
Out[9]: array([False,  True,  True,  True], dtype=bool)
In [10]: 0 < np.arange(4)
Out[10]: array([False,  True,  True,  True], dtype=bool)

Right now these comparisons will throw error. With the proposed fix, these can be expected to work fine.

In [3]: pd.Series(np.arange(1)) < pd.Series(np.arange(4))
In [4]: np.arange(1) < pd.Series(np.arange(4))
In [5]: pd.Series(np.arange(4)) > pd.Series(np.arange(1))
In [6]: pd.Series(np.arange(4)) > np.arange(1)

[wadawson]

I can confirm @dmsul 's diagnosis of this bug. Afraid I don't have time currently to correct this but +1 for a fix.