Skip to content

Datetimelike index / arrays ignore integer sign and size in astype #24381

Closed
@TomAugspurger

Description

@TomAugspurger

For datetimelike indexes, .astype ignore the sign and size of an integer dtype, and always uses int64

In [1]: import pandas as pd

In [2]: idx = pd.date_range('2000', periods=4)

In [3]: idx.astype('uint16')
Out[3]:
Int64Index([946684800000000000, 946771200000000000, 946857600000000000,
            946944000000000000],
           dtype='int64')

We don't have Index subclasses for each of these types (though we do have uint64 now).

In #24024 we agreed(?) that following the Index was the best behavior for the array class.

In [7]: idx._data.astype('uint16')
Out[7]:
array([946684800000000000, 946771200000000000, 946857600000000000,
       946944000000000000])

(that's also int64).


Some questions:

  1. Do we want to change this? Note that for "normal" usage of datetimes, this is going to truncate / overflow. This is particularly unfriendly to 32-bit systems, where "int" is "int32".
In [9]: idx._data.astype('int32').astype("int32")
Out[9]: array([1380122624, -476971008, 1960902656,  103809024], dtype=int32)

So maybe we would want safe casting?

In [10]: idx._data.astype('int32').astype("int32", casting="safe")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-2c2a4a677a5c> in <module>
----> 1 idx._data.astype('int32').astype("int32", casting="safe")

TypeError: Cannot cast array from dtype('int64') to dtype('int32') according to the rule 'safe'
  1. ExtensionIndex classes are in the not too distant future, so we may be able to relatively easily get Index classes that work with these different integer sizes / signs. So we might be able to properly support this in the future (if we want to).

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugDatetimeDatetime data dtypeDtype ConversionsUnexpected or buggy dtype conversions

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions