Skip to content

BUG: Timedelta drops decimals if precision is greater than nanoseconds #36771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Nov 15, 2020
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ Datetimelike
- Bug in :class:`DatetimeIndex.shift` incorrectly raising when shifting empty indexes (:issue:`14811`)
- :class:`Timestamp` and :class:`DatetimeIndex` comparisons between timezone-aware and timezone-naive objects now follow the standard library ``datetime`` behavior, returning ``True``/``False`` for ``!=``/``==`` and raising for inequality comparisons (:issue:`28507`)
- Bug in :meth:`DatetimeIndex.equals` and :meth:`TimedeltaIndex.equals` incorrectly considering ``int64`` indexes as equal (:issue:`36744`)
- Bug in :class:`Timedelta` incorrectly deleted all decimals when input had a higher precision than nanoseconds (:issue:`36738`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"incorrectly truncating to sub-second portion of a string input when it has precision higher than nanoseconds"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Timedelta
^^^^^^^^^
Expand Down
9 changes: 7 additions & 2 deletions pandas/_libs/tslibs/timedeltas.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -404,9 +404,11 @@ cdef inline int64_t parse_timedelta_string(str ts) except? -1:
m = 10**(3 -len(frac)) * 1000 * 1000
elif len(frac) > 3 and len(frac) <= 6:
m = 10**(6 -len(frac)) * 1000
else:
elif len(frac) > 6 and len(frac) <= 9:
m = 10**(9 -len(frac))

else:
m = 1
frac = frac[:9]
r = <int64_t>int(''.join(frac)) * m
result += timedelta_as_neg(r, neg)

Expand Down Expand Up @@ -1132,6 +1134,9 @@ class Timedelta(_Timedelta):
Notes
-----
The ``.value`` attribute is always in ns.

If the precision is higher than nanoseconds, the precision of the duration is
truncated to nanoseconds.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarify this is for a string input

do additional digits round or just drop (and if just drop, are we sure thats the desired behavior?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropd additional digits. Was not sure either so I started with the easier implementation
#36771 (comment)

"""

def __new__(cls, object value=_no_input, unit=None, **kwargs):
Expand Down
6 changes: 6 additions & 0 deletions pandas/core/tools/timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,12 @@ def to_timedelta(arg, unit=None, errors="raise"):
to_datetime : Convert argument to datetime.
convert_dtypes : Convert dtypes.

Notes
-----

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no extra line here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx, changed

If the precision is higher than nanoseconds, the precision of the duration is
truncated to nanoseconds.

Examples
--------
Parsing a single string to a Timedelta:
Expand Down
7 changes: 7 additions & 0 deletions pandas/tests/tools/test_to_timedelta.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,3 +179,10 @@ def test_to_timedelta_nullable_int64_dtype(self):
result = to_timedelta(Series([1, None], dtype="Int64"), unit="days")

tm.assert_series_equal(result, expected)

@pytest.mark.parametrize("func", ["Timedelta", "to_timedelta"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use pd.Timedelta and pd.to_datetime instead of passing these as strings and using getattr below

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

def test_to_timedelta_precision_over_nanos(self, func):
# GH: 36738
expected = pd.Timedelta("8:53:08.718")
result = getattr(pd, func)("8:53:08.71800000001")
assert result == expected