Skip to content

BUG: Interchange with PyArrow types does not match PyArrow behavior #56801

Open
@WillAyd

Description

@WillAyd

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>> import datetime
>>> import pandas as pd
>>> df = pd.DataFrame([[datetime.date(2023, 1, 1)]], dtype=pd.ArrowDtype(pa.date32()))
>>> df.dtypes
0    date32[day][pyarrow]
dtype: object

>>> df.__dataframe__().get_column(0).get_buffers()
{'data': (PandasBuffer({'bufsize': 8, 'ptr': 103818415071840, 'device': 'CPU'}), (<DtypeKind.DATETIME: 22>, 64, 'tdD', '=')), 'validity': None, 'offsets': None}

>>> pa.Table.from_pandas(df).__dataframe__().get_column(0).get_buffers()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/willayd/mambaforge/envs/pantab-dev/lib/python3.12/site-packages/pyarrow/interchange/dataframe.py", line 139, in get_column
    return _PyArrowColumn(self._df.column(i),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/willayd/mambaforge/envs/pantab-dev/lib/python3.12/site-packages/pyarrow/interchange/column.py", line 239, in __init__
    self._dtype = self._dtype_from_arrowdtype(dtype, bit_width)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/willayd/mambaforge/envs/pantab-dev/lib/python3.12/site-packages/pyarrow/interchange/column.py", line 322, in _dtype_from_arrowdtype
    raise ValueError(
ValueError: Data type date32[day] not supported by interchange protocol

Issue Description

I find it surprising that pandas provides a different value to the interchange protocol for an Arrow dtype than Arrow itself would (side note, I am also surprised Arrow fails here)

cc @MarcoGorelli

Expected Behavior

Should match arrow

Installed Versions

main

Metadata

Metadata

Assignees

No one assigned

    Labels

    Arrowpyarrow functionalityBugInterchangeDataframe Interchange Protocol

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions