Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the master branch of pandas.
Reproducible Example
import pandas as pd
import numpy as np
df = pd.DataFrame(
{"a": [1,1,1,np.NaN],
"b": [1,1,2,np.NaN,],
"c": ["a", "b", "c", "d"]})
print(df.groupby(["a", "b"], dropna=False)[["c"]].agg(["unique", "nunique"]))
print(df.groupby(["a", "b"], dropna=False)[["c"]].agg(["nunique", "unique"]))
Issue Description
As you can see from he example, if I use the agg
function with nunique
before unique
, the row with the NaNs appears duplicated:
Correct:
print(df.groupby(["a", "b"], dropna=False)[["c"]].agg(["unique", "nunique"]))
c
unique nunique
a b
1.0 1.0 [a, b] 2
2.0 [c] 1
NaN NaN [d] 1
Incorrect
print(df.groupby(["a", "b"], dropna=False)[["c"]].agg(["nunique", "unique"]))
c
nunique unique
a b
1.0 1.0 2 [a, b]
2.0 1 [c]
NaN NaN 1 [d]
NaN 1 [d]
This only happens with NaN
s, if I fillna(0)
everything works as expected.
Expected Behavior
The order of aggregation functions should not impact the result and rows should not be duplicated.
Installed Versions
INSTALLED VERSIONS
commit : 66e3805
python : 3.10.1.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-147-generic
Version : #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.3.5
numpy : 1.22.0
pytz : 2021.3
dateutil : 2.8.2
pip : 21.3.1
setuptools : 60.2.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.3
IPython : 7.30.1
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.5.1
numexpr : None
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.7.3
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None