Skip to content

BUG: Logical and comparison ops with ArrowDtype & masked #52633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Apr 18, 2023
Merged

BUG: Logical and comparison ops with ArrowDtype & masked #52633

merged 3 commits into from
Apr 18, 2023

Conversation

mroeschke
Copy link
Member

@mroeschke mroeschke commented Apr 12, 2023 •
edited
Loading

@mroeschke mroeschke added NA - MaskedArrays Related to pd.NA and nullable extension arrays Arrow pyarrow functionality labels Apr 12, 2023
@mroeschke mroeschke added this to the 2.0.1 milestone Apr 12, 2023
jbrockmendel
jbrockmendel reviewed Apr 12, 2023
View reviewed changes
pandas/core/arrays/arrow/array.py
@@ -451,6 +452,9 @@ def _cmp_method(self, other, op):
result = pc_func(self._pa_array, other._pa_array)
elif isinstance(other, (np.ndarray, list)):
result = pc_func(self._pa_array, other)
elif isinstance(other, BaseMaskedArray):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might this be a use case for __pandas_priority__?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm might this unintentionally make ArrowExtentionArray be prioritized over a custom 3rd party array?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1000-1999 are for EAs. I guess we could document that internal EAs only use up through e.g. 1499

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding of __pandas_priority__ was that it was specifically for non-internal EAs since at least for the internal ones we can dictate the return. Or did you imagine we are supposed to be using this internally as well?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internally it doesn't really matter which method we used. i was imagining we could use it internally for intra-EA ops but that wasn't a big selling point

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this check and using __pandas_priority__ are essentially similar I would slightly prefer to use this check to avoid spilling into interacting with 3rd party EA.pandas_priority unless you feel strongly otherwise

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No objection

@mroeschke mroeschke merged commit c85bbc6 into pandas-dev:main Apr 18, 2023
@mroeschke mroeschke deleted the bug/comp/arrow_masked branch April 18, 2023 19:59
@lumberbot-app
Copy link

lumberbot-app bot commented Apr 18, 2023

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 2.0.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 c85bbc6913b9c4d2acc53c54cc858c84bf0658d1
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #52633: BUG: Logical and comparison ops with ArrowDtype & masked'
  1. Push to a named branch:
git push YOURFORK 2.0.x:auto-backport-of-pr-52633-on-2.0.x
  1. Create a PR against branch 2.0.x, I would have named this PR:

"Backport PR #52633 on branch 2.0.x (BUG: Logical and comparison ops with ArrowDtype & masked)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

mroeschke added a commit to mroeschke/pandas that referenced this pull request Apr 18, 2023
@mroeschke
Backport PR pandas-dev#52633: BUG: Logical and comparison ops with Ar…
4f362f7 
…rowDtype & masked
mroeschke added a commit that referenced this pull request Apr 19, 2023
@mroeschke
Backport PR #52633 on branch 2.0.x (BUG: Logical and comparison ops w…
5a95983 
…ith ArrowDtype & masked) (#52767)

* Backport PR #52633: BUG: Logical and comparison ops with ArrowDtype & masked

* Make runtime import
@AlenkaF AlenkaF mentioned this pull request Apr 19, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbrockmendel jbrockmendel jbrockmendel left review comments

Assignees
No one assigned
Labels
Arrow pyarrow functionality NA - MaskedArrays Related to pd.NA and nullable extension arrays
Projects
None yet
Milestone
2.0.1
Development

Successfully merging this pull request may close these issues.

BUG: ArrowExtensionArray logical_op not working in all directions
2 participants
@mroeschke @jbrockmendel