DOC: `.str.cat` output in case of `Index` object

[galipremsagar]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import pandas as pd
import numpy as np
i = pd.Index(["AbC", "de", "FGHI", "j", "kLm"])
other = (
            pd.Series(["f", "g", "h", "i", "j"]),
            np.array(["f", "a", "b", "f", "a"]),
            pd.Series(["f", "g", "h", "i", "j"]),
            np.array(["f", "a", "b", "f", "a"]),
            np.array(["f", "a", "b", "f", "a"]),
            pd.Index(["1", "2", "3", "4", "5"]),
            np.array(["f", "a", "b", "f", "a"]),
            pd.Index(["f", "g", "h", "i", "j"]),
)
print(i.str.cat(other))
x = pd.Series(["f", "g", "h", "i", "j"])
print(i.str.cat(x))

0.25.3 output:

Index(['AbCfffff1ff', 'degagaa2ag', 'FGHIhbhbb3bh', 'jififf4fi',
       'kLmjajaa5aj'],
      dtype='object')
Index(['AbCf', 'deg', 'FGHIh', 'ji', 'kLmj'], dtype='object')

1.1.0 output:

Index([nan, nan, nan, nan, nan], dtype='object')
Index([nan, nan, nan, nan, nan], dtype='object')

Problem description

The output of 0.25.3 version seems to be correct as when we change i to be a Series the incorrect results go away:

import pandas as pd
import numpy as np
i = pd.Series(["AbC", "de", "FGHI", "j", "kLm"])
other = (
            pd.Series(["f", "g", "h", "i", "j"]),
            np.array(["f", "a", "b", "f", "a"]),
            pd.Series(["f", "g", "h", "i", "j"]),
            np.array(["f", "a", "b", "f", "a"]),
            np.array(["f", "a", "b", "f", "a"]),
            pd.Index(["1", "2", "3", "4", "5"]),
            np.array(["f", "a", "b", "f", "a"]),
            pd.Index(["f", "g", "h", "i", "j"]),
)
print(i.str.cat(other))
x = pd.Series(["f", "g", "h", "i", "j"])
print(i.str.cat(x))


0     AbCfffff1ff
1      degagaa2ag
2    FGHIhbhbb3bh
3       jififf4fi
4     kLmjajaa5aj
dtype: object
0     AbCf
1      deg
2    FGHIh
3       ji
4     kLmj
dtype: object

Expected Output

The output should be similar to Series behaviour/ 0.25.3 behaviour.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : d9fff27
python : 3.7.3.final.0
python-bits : 64
OS : Darwin
OS-release : 19.6.0
Version : Darwin Kernel Version 19.6.0: Sun Jul 5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : en_US.UTF-8
pandas : 1.1.0
numpy : 1.19.0
pytz : 2020.1
dateutil : 2.8.1
pip : 20.1.1
setuptools : 49.1.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[dsaxton]

I think this is expected as per the warning from 0.25.3 (that the operation is first going to perform index alignment in a future version):

In [1]: import pandas as pd                                                                                                                                                                                  
In [2]: idx = pd.Index(["a", "b", "c", "d", "e"])                                                                                                                                                            
In [3]: ser = pd.Series(["f", "g", "h", "i", "j"])                                                                                                                                                           
In [4]: print(idx.str.cat(ser))                                                                                                                                                                              
<ipython-input-4-157619096d44>:1: FutureWarning: A future version of pandas will perform index alignment when `others` is a Series/Index/DataFrame (or a list-like containing one). To disable alignment (the behavior before v.0.23) and silence this warning, use `.values` on any Series/Index/DataFrame in `others`. To enable alignment and silence this warning, pass `join='left'|'outer'|'inner'|'right'`. The future default will be `join='left'`.
  print(idx.str.cat(ser))
/Users/danielsaxton/opt/miniconda3/envs/pandas-0.25.3/lib/python3.8/site-packages/numpy/core/fromnumeric.py:87: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
Index(['af', 'bg', 'ch', 'di', 'ej'], dtype='object')

In [5]: print(pd.__version__)                                                                                                                                                                                
0.25.3

Following the suggestion from the warning and casting to numpy first we get the desired output:

In [1]: import pandas as pd                                                                                                                                                                                  
In [2]: idx = pd.Index(["a", "b", "c", "d", "e"])                                                                                                                                                            
In [3]: ser = pd.Series(["f", "g", "h", "i", "j"])                                                                                                                                                           
In [4]: print(idx.str.cat(ser))                                                                                                                                                                              
Index([nan, nan, nan, nan, nan], dtype='object')

In [5]: print(pd.__version__)                                                                                                                                                                                
1.2.0.dev0+29.ga4203cf8d

In [6]: idx.str.cat(ser.to_numpy())                                                                                                                                                                          
Out[6]: Index(['af', 'bg', 'ch', 'di', 'ej'], dtype='object')

After taking a look I'm not sure if this is explicit enough in the documentation.

[simonjayhawkins]

Thanks @galipremsagar for the report and thanks @dsaxton for the answer.

After taking a look I'm not sure if this is explicit enough in the documentation.

would take documentation improvements so will leave this issue open

[souris-dev]

Hello, I would like to work on this issue.