Skip to content

BUG: Left join is broken with a MultiIndex with only one level #36909

Closed
@theemathas

Description

@theemathas
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

import pandas as pd

foo = pd.DataFrame(data={'c': 3}, index=pd.MultiIndex.from_tuples([(1, 2)], names=('a', 'b')))
bar = pd.DataFrame(data={'d': 4}, index=pd.MultiIndex.from_tuples([(2,)], names=('b',)))
joined = foo.join(bar, how='left')
print(joined)

Expected Output

     c d
b a       
2 1  3 4

(Throwing an exception on the line where bar is created is also possibly acceptable, although confusing.)

Produced Output

     c   d
b a       
2 1  3 NaN

Problem description

When joining with a data frame with a MultiIndex with a single level in it, left joins seem to not work properly. This behavior is confusing (why would a MultiIndex with a single index not work?), and is inconsistent with similar code which produces expected output.

Possibly related issues: #34292, #29252, #34357

Examples of similar code with expected behavior

Example 1:

import pandas as pd

foo = pd.DataFrame(data={'c': 3}, index=pd.MultiIndex.from_tuples([(1, 2)], names=('a', 'b')))
bar = pd.DataFrame(data={'d': 4}, index=pd.Index([2], name='b'))
joined = foo.join(bar, how='left')
print(joined)

produces

     c  d
a b      
1 2  3  4

Example 2:

import pandas as pd

foo = pd.DataFrame(data={'c': 3}, index=pd.MultiIndex.from_tuples([(1, 2, 99)], names=('a', 'b', 'x')))
bar = pd.DataFrame(data={'d': 4}, index=pd.MultiIndex.from_tuples([(2, 99)], names=('b', 'x')))
joined = foo.join(bar, how='left')
print(joined)

produces

        c  d
b x  a      
2 99 1  3  4

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : db08276bc116c438d3fdee492026f8223584c477
python           : 3.8.6.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
Version          : 10.0.18362
machine          : AMD64
processor        : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : English_United States.1252

pandas           : 1.1.3
numpy            : 1.19.2
pytz             : 2020.1
dateutil         : 2.8.1
pip              : 20.1.1
setuptools       : 50.3.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fsspec           : None
fastparquet      : None
gcsfs            : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pyxlsb           : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
numba            : None

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions