Skip to content

merge_asof() tolerance cannot be datetime.timedelta  #28098

Closed
@ianzur

Description

@ianzur

Code Sample

import pandas as pd
import numpy as np
from datetime import timedelta

left = pd.DataFrame(
    list(zip([0, 5, 10, 15, 20, 25], [0, 1, 2, 3, 4, 5])),
    columns=["time", "left"],
)
left["time"] = pd.to_timedelta(left["time"], "ms")

right = pd.DataFrame(
    list(zip([0, 3, 9, 12, 15, 18], [0, 1, 2, 3, 4, 5])),
    columns=["time", "right"],
)
right["time"] = pd.to_timedelta(right["time"], "ms")

expected = pd.DataFrame(
    list(
        zip(
            [0, 5, 10, 15, 20, 25],
            [0, 1, 2, 3, 4, 5],
            [0, np.nan, 2, 4, np.nan, np.nan],
        )
    ),
    columns=["time", "left", "right"],
)
expected["time"] = pd.to_timedelta(expected["time"], "ms")
print(expected)

# this throws MergeError: incompatible tolerance <class 'datetime.timedelta'>, must be compat with type dtype('<m8[ns]')
result = pd.merge_asof(left, right, on="time", tolerance=timedelta(microseconds=1000), direction="nearest")

# this works!
# result = pd.merge_asof(left, right, on="time", tolerance=pd.Timedelta("1ms"), direction="nearest")

print(result)

Problem description

I can use pd.Timedelta as tolerance, but not datetime.timedelta.

Expected Output

No error

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.7.3.final.0 python-bits : 64 OS : Linux OS-release : 4.19.0-5-amd64 machine : x86_64 processor : byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8

pandas : 0.25.0+222.gdef01cf7b
numpy : 1.17.0
pytz : 2019.2
dateutil : 2.8.0
pip : 19.2.2
setuptools : 41.2.0
Cython : 0.29.13
pytest : 5.1.1
hypothesis : 4.33.1
sphinx : 1.8.5
blosc : 1.8.1
feather : None
xlsxwriter : 1.1.9
lxml.etree : 4.4.1
html5lib : 1.0.1
pymysql : None
psycopg2 : None
jinja2 : 2.10.1
IPython : 7.7.0
pandas_datareader: None
bs4 : 4.8.0
bottleneck : 1.2.1
fastparquet : 0.3.2
gcsfs : None
lxml.etree : 4.4.1
matplotlib : 3.1.1
numexpr : 2.7.0
odfpy : None
openpyxl : 2.6.3
pandas_gbq : None
pyarrow : 0.14.1
pytables : None
s3fs : 0.3.3
scipy : 1.3.1
sqlalchemy : 1.3.7
tables : 3.5.2
xarray : 0.12.3
xlrd : 1.2.0
xlwt : 1.3.0
xlsxwriter : 1.1.9

Metadata

Metadata

Assignees

No one assigned

    Labels

    ReshapingConcat, Merge/Join, Stack/Unstack, ExplodeTimedeltaTimedelta data type

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions