Skip to content

BUG: ArrowDtype.numpy_dtype returning ns units for non-nano timestamp and duration types #51800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 6, 2023

Conversation

lukemanley
Copy link
Member

ArrowDtype.numpy_dtype goes through the pyarrow to_pandas_type method which was added before pandas supported non-nano units. This PR ensures the correct units are passed along to the numpy dtype. This fixes 6 xfails.

pyarrow already preserves the units when going directly from pyarrow to numpy:

>>> import pyarrow as pa
>>> arr = pa.array([1, 2, 3], type=pa.duration("us"))
>>> arr.to_numpy()
array([1, 2, 3], dtype='timedelta64[us]')

@lukemanley lukemanley added Bug Arrow pyarrow functionality Non-Nano datetime64/timedelta64 with non-nanosecond resolution labels Mar 5, 2023
@mroeschke mroeschke added this to the 2.1 milestone Mar 6, 2023
@jbrockmendel jbrockmendel merged commit fa01870 into pandas-dev:main Mar 6, 2023
@jbrockmendel
Copy link
Member

thanks @lukemanley

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Arrow pyarrow functionality Bug Non-Nano datetime64/timedelta64 with non-nanosecond resolution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants