API/BUG: obj.str.slice(start:None:-1)

[jbrockmendel]

import pandas as pd
import pyarrow as pa

ser = pd.Series(["aafootwo", "aabartwo", pd.NA, "aabazqux"], dtype="string[pyarrow]")
ser2 = ser.astype(pd.ArrowDtype(pa.string()))

>>> ser.str.slice(None, None, -1)
0    owtoofaa
1    owtrabaa
2        <NA>
3    xuqzabaa
dtype: string

>>> ser2.str.slice(None, None, -1)
0       a
1       a
2    <NA>
3       a
dtype: string[pyarrow]

This looks to me like the ArrowEA version is wrong, can @mroeschke or @jorisvandenbossche confirm this is not intentional?

The ArrowEA version's only test is in test_str_slice in tests/extension/test_arrow.py, has no test cases with either stop=None or step<0. The StringDtype version has a test_slice in tests/strings/test_strings.py that has a bit better coverage.

[mroeschke]

Looks like ArrowStringArray falls back for the stop=None case compared to ArrowEA.

Both dispatch to https://arrow.apache.org/docs/python/generated/pyarrow.compute.utf8_slice_codeunits.html though, and I don't know if it's supposed to support negative steps. If not, I would say the ArrowEA._str_slice has a bug and it should have logic to swap the start and stop based on the sign on step

[jorisvandenbossche]

I don't know if it's supposed to support negative steps.

According to the docstring you linked, it does. Trying it out, if you pass explicit start/stop it seems to work:

In [15]: pc.utf8_slice_codeunits(["aafootwo"], start=5, stop=3, step=-1)
Out[15]: 
<pyarrow.lib.StringArray object at 0x7fd9cd9ce200>
[
  "to"
]

In [16]: "aafootwo"[5:3:-1]
Out[16]: 'to'

So it might be our translation of the passed user-passed arguments to pyarrow. Because with pyarrow, you need to specify some start value, you cannot do slice(none, None, -1) as in python. But explicitly saying to start at the end in that case seems to work again:

In [18]: "aafootwo"[None:None:-1]
Out[18]: 'owtoofaa'

In [19]: pc.utf8_slice_codeunits(["aafootwo"], start=-1, stop=None, step=-1)
Out[19]: 
<pyarrow.lib.StringArray object at 0x7fd9d7eadd20>
[
  "owtoofaa"
]