Skip to content

BUG: ArrowExtensionArray logical_op not working in all directions #52625

Closed
@MCRE-BE

Description

@MCRE-BE

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

a = pd.Series(
    data = [True, True, False],
    dtype = pd.BooleanDtype()
)
b = pd.Series(
    data = [True, True, False],
    dtype = "boolean[pyarrow]"
)

a & b
0     True
1     True
2    False
dtype: boolean

b & a
NotImplementedError                       Traceback (most recent call last)
Cell In[209], line 1
----> 1 b & a

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\ops\common.py:81, in _unpack_zerodim_and_defer..new_method(self, other)
     77             return NotImplemented
     79 other = item_from_zerodim(other)
---> 81 return method(self, other)

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\arraylike.py:70, in OpsMixin.__and__(self, other)
     68 @unpack_zerodim_and_defer("__and__")
     69 def __and__(self, other):
---> 70     return self._logical_method(other, operator.and_)

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\series.py:6108, in Series._logical_method(self, other, op)
   6105 lvalues = self._values
   6106 rvalues = extract_array(other, extract_numpy=True, extract_range=True)
-> 6108 res_values = ops.logical_op(lvalues, rvalues, op)
   6109 return self._construct_result(res_values, name=res_name)

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\ops\array_ops.py:379, in logical_op(left, right, op)
    375 rvalues = right
    377 if should_extension_dispatch(lvalues, rvalues):
    378     # Call the method on lvalues
--> 379     res_values = op(lvalues, rvalues)
    381 else:
    382     if isinstance(rvalues, np.ndarray):

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\ops\common.py:81, in _unpack_zerodim_and_defer..new_method(self, other)
     77             return NotImplemented
     79 other = item_from_zerodim(other)
---> 81 return method(self, other)

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\arraylike.py:70, in OpsMixin.__and__(self, other)
     68 @unpack_zerodim_and_defer("__and__")
     69 def __and__(self, other):
---> 70     return self._logical_method(other, operator.and_)

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\arrays\arrow\array.py:503, in ArrowExtensionArray._logical_method(self, other, op)
    502 def _logical_method(self, other, op):
--> 503     return self._evaluate_op_method(other, op, ARROW_LOGICAL_FUNCS)

File c:\ProgramData\Miniconda3\envs\DLL_ETL\lib\site-packages\pandas\core\arrays\arrow\array.py:497, in ArrowExtensionArray._evaluate_op_method(self, other, op, arrow_funcs)
    495     result = pc_func(self._data, pa_scalar)
    496 else:
--> 497     raise NotImplementedError(
    498         f"{op.__name__} not implemented for {type(other)}"
    499     )
    500 return type(self)(result)
NotImplementedError: and_ not implemented for

Issue Description

Logical operations don't work in all directions.
Issue seems to be pd.BooleanDtype() as it works for

  • bool & pyarrow[boolean] : both directions
  • bool & pd.BooleanDtype : both directions
  • pd.BooleanDtype & pyarrow[boolean] : 👍
  • pyarrow[boolean] & pd.BooleanDtype : 👎

Tested for And / Or

Expected Behavior

Should work

Installed Versions

INSTALLED VERSIONS

commit : 478d340
python : 3.10.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19042
machine : AMD64
processor : AMD64 Family 23 Model 24 Stepping 1, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_Netherlands.1252

pandas : 2.0.0
numpy : 1.23.5
pytz : 2023.3
dateutil : 2.8.2
setuptools : 67.6.1
pip : 23.0.1
Cython : None
pytest : 7.2.2
hypothesis : None
sphinx : 6.1.3
blosc : None
feather : None
xlsxwriter : 3.0.9
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.12.0
pandas_datareader: 0.10.0
bs4 : 4.12.1
bottleneck : None
brotli :
fastparquet : None
fsspec : 2023.3.0
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : 3.1.0
pandas_gbq : None
pyarrow : 11.0.0
pyreadstat : None
pyxlsb : 1.0.10
s3fs : None
scipy : 1.10.1
snappy : None
sqlalchemy : None
tables : None
tabulate : 0.9.0
xarray : None
xlrd : 2.0.1
zstandard : 0.19.0
tzdata : 2023.3
qtpy : 2.3.1
pyqt5 : None

Metadata

Metadata

Assignees

Labels

Arrowpyarrow functionalityBug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions