Avoid listifying left/right operand in ExtensionArray ops dispatching

[jorisvandenbossche]

We now have the follow code in the ExtensionArray ops dispatching:

pandas/pandas/core/ops.py

Lines 1169 to 1192 in 2f1b842

	# TODO(jreback)
	# we need to listify to avoid ndarray, or non-same-type extension array
	# dispatching
	if is_extension_array_dtype(left):
	new_left = left.values
	if isinstance(right, np.ndarray):
	# handle numpy scalars, this is a PITA
	# TODO(jreback)
	new_right = lib.item_from_zerodim(right)
	if is_scalar(new_right):
	new_right = [new_right]
	new_right = list(new_right)
	elif is_extension_array_dtype(right) and type(left) != type(right):
	new_right = list(new_right)
	else:
	new_right = right
	else:
	new_left = list(left.values)
	new_right = right

There are a few places where we convert the values to a list, which should be avoided.

[TomAugspurger]

This specific case of Series[EA] + EA (where the EAs are the same type) seems very easy to handle.

[jorisvandenbossche]

Looking at the code, it seems that that case is already working OK, if the type(EA) is equal.

[TomAugspurger]

Hmm is it? I thought that's what cased the NameError. I think we need something like type(new_left) is type(right), where new_left has unboxed the EA from the Series. Haven't looked closely though.

[jorisvandenbossche]

Yes, it's what caused the NameError, but this is about the conversion to list?

[TomAugspurger]

Correct. I'm saying we shouldn't even reach that elif in the case of Series[EA] + EA.

elif is_extension_array_dtype(right) and type(new_left) == type(right):
    # EA + EA. This is new.
    new_right = right
elif is_extension_array_dtype(right) and type(left) != type(right):
    new_right = list(right)

[TomAugspurger]

Corner case in pandas master with object-dtype

In [1]: import pandas as pd

In [2]: 1 ** pd.Series([None], dtype=object)
Out[2]:
0    NaN
dtype: object

That's supposed to be 1, since 1 ** anything is 1

In [4]: 1 ** pd.Series([np.nan])
Out[4]:
0    1.0
dtype: float64

[jorisvandenbossche]

@TomAugspurger is that related to the listifying?

[TomAugspurger]

Whoops, got my issues mixed up. That was for #22022