Skip to content

Merging on int / float columns with NaNs gives spurious warning #20998

Closed
@jorisvandenbossche

Description

@jorisvandenbossche

In 0.23 we added a warning when trying to merge on int / float columns where the float values are not actually integers (#18352). No objections to that, but I think we should special case for NaNs, as this is a typical case that turns an integer column into floats, but does give the warning:

In [8]: df1 = pd.DataFrame({'col1': [0, 1], 'key': [1, 2]})
   ...: df2 = pd.DataFrame({'col2': ['a', 'b'], 'key': [2, np.nan]})

In [9]: pd.merge(df1, df2, on='key')
/home/joris/scipy/pandas/pandas/core/reshape/merge.py:969: UserWarning: You are merging on int and float columns where the float values are not equal to their int representation
  'representation', UserWarning)
Out[9]: 
   col1  key col2
0     1    2    a

So I think the above should not generate the warning.
A fix is probably to just mask for missing values in the check (PR coming).

Metadata

Metadata

Assignees

No one assigned

    Labels

    ReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions