TypeError on argmax of object dtype (change from 0.20.3)

[keerthanpg]

>>> import pandas as pd
>>> pd.Series([0, 0], dtype='object').argmax()

---

I was doing action = state_action.idxmax() where state_action was of type 'pandas.core.series.Series'. When I run in 0.21.0, it gives the following error:

File "/usr/local/lib/python3.5/dist-packages/pandas/core/series.py", line 1357, in idxmax
i = nanops.nanargmax(_values_from_object(self), skipna=skipna)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/nanops.py", line 74, in _f
raise TypeError(msg.format(name=f.name.replace('nan', '')))
TypeError: reduction operation 'argmax' not allowed for this dtype

However, when I downgraded to pandas 0.20.3, it worked just fine. You might wanna look into this. :)

[TomAugspurger]

Can you give a reproducible example?

[barondu]

I meet the same problem.
At first i use action = state_action.argmax(), it saysFutureWarning: 'argmax' is deprecated. Use 'idxmax' instead. The behavior of 'argmax' will be corrected to return the positional maximum in the future. Use 'series.values.argmax' to get the position of the maximum now. action = state_action.argmax()
So I change to action = state_action.idxmax()
When I run in 0.21.0, it gives the following error:

Traceback (most recent call last):
  File "/Users/baron/.pyenv/versions/3.6.3/lib/python3.6/tkinter/__init__.py", line 1699, in __call__
    return self.func(*args)
  File "/Users/baron/.pyenv/versions/3.6.3/lib/python3.6/tkinter/__init__.py", line 745, in callit
    func(*args)
  File "/Users/baron/PycharmProjects/HelloPython/test_Q.py", line 26, in update
    action = RL.choose_action(str(observation))
  File "/Users/baron/PycharmProjects/HelloPython/RL_brain.py", line 40, in choose_action
    action = state_action.idxmax()
  File "/Users/baron/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pandas/core/series.py", line 1357, in idxmax
    i = nanops.nanargmax(_values_from_object(self), skipna=skipna)
  File "/Users/baron/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pandas/core/nanops.py", line 74, in _f
    raise TypeError(msg.format(name=f.__name__.replace('nan', '')))
TypeError: reduction operation 'argmax' not allowed for this dtype

[TomAugspurger]

Can you provide a copy-pastable example @barondu?

[barondu]

sure,
You can test the code as fellow. @TomAugspurger
https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow/tree/master/contents/2_Q_Learning_maze

[TomAugspurger]

Do you have a minimal test-case, something that could go in a unit test?

[barondu]

@TomAugspurger

import pandas as pd
import numpy as np

q_table = pd.DataFrame(columns=['a', 'b', 'c', 'd'])
q_table = q_table.append(pd.Series([0] * 4, index=q_table.columns, name='test1', ))
q_table = q_table.append(pd.Series([0] * 4, index=q_table.columns, name='test2', ))
print(q_table)
state_action = q_table.ix['test2', :]
print(state_action)
state_action = state_action.reindex(
    np.random.permutation(state_action.index))
print(state_action)
action = state_action.idxmax()
# action = state_action.argmax()
print('\naction: ', action)

[barondu]

Here is the error message

Traceback (most recent call last):
  File "/Users/baron/PycharmProjects/HelloPython/pandas_exercise.py", line 13, in <module>
    action = state_action.idxmax()
  File "/Users/baron/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pandas/core/series.py", line 1357, in idxmax
    i = nanops.nanargmax(_values_from_object(self), skipna=skipna)
  File "/Users/baron/.pyenv/versions/3.6.3/lib/python3.6/site-packages/pandas/core/nanops.py", line 74, in _f
    raise TypeError(msg.format(name=f.__name__.replace('nan', '')))
TypeError: reduction operation 'argmax' not allowed for this dtype

[TomAugspurger]

Thanks, simplified a bit:

In [11]: pd.Series([0, 0], dtype='object')
Out[11]:
0    0
1    0
dtype: object

In [12]: pd.Series([0, 0], dtype='object').argmax()
/Users/taugspurger/Envs/pandas-dev/bin/ipython:1: FutureWarning: 'argmax' is deprecated. Use 'idxmax' instead. The behavior of 'argmax' will be corrected to return the positional maximum in the future. Use 'series.values.argmax' to get the position of the maximum now.
  #!/Users/taugspurger/Envs/pandas-dev/bin/python3.6
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-e0ba19c8565d> in <module>()
----> 1 pd.Series([0, 0], dtype='object').argmax()

~/Envs/pandas-dev/lib/python3.6/site-packages/pandas/pandas/util/_decorators.py in wrapper(*args, **kwargs)
     34     def wrapper(*args, **kwargs):
     35         warnings.warn(msg, klass, stacklevel=stacklevel)
---> 36         return alternative(*args, **kwargs)
     37     return wrapper
     38

~/Envs/pandas-dev/lib/python3.6/site-packages/pandas/pandas/core/series.py in idxmax(self, axis, skipna, *args, **kwargs)
   1355         """
   1356         skipna = nv.validate_argmax_with_skipna(skipna, args, kwargs)
-> 1357         i = nanops.nanargmax(_values_from_object(self), skipna=skipna)
   1358         if i == -1:
   1359             return np.nan

~/Envs/pandas-dev/lib/python3.6/site-packages/pandas/pandas/core/nanops.py in _f(*args, **kwargs)
     72             if any(self.check(obj) for obj in obj_iter):
     73                 msg = 'reduction operation {name!r} not allowed for this dtype'
---> 74                 raise TypeError(msg.format(name=f.__name__.replace('nan', '')))
     75             try:
     76                 with np.errstate(invalid='ignore'):

TypeError: reduction operation 'argmax' not allowed for this dtype

Is there a reason you're using object dtype here?

[TomAugspurger]

Seems like #16449 maybe have been the root issues (cc @DGrady)

NumPy will (somehow) handle object arrays in argmax/min, so I suppose @disallow('O') is a bit too strict.