agg behaviour depends on the number of arguments of the function

[bertrandmarc]

Code Sample, a copy-pastable example if possible

import pandas as pd

df = pd.DataFrame({'a': [1,2,3,4,5,6],
 'b': [1,1,0,1,1,0],
 'c': ['x','x','x','z','z','z'],
 'd': ['s','s','s','d','d','d']})

def myprint2(x, y):
    print(x)

df.groupby(['c'])[['a', 'b']].agg(myprint2, 0)

Problem description

In the particular case of a groupby over 1 column ('c'), asking a subset of columns ('a' and 'b') and passing agg a function of 2 arguments ('mypring2'), a DataFrame of the full groups (including all columns) is passed to myprint2:

>>>> df.groupby(['c'])[['a', 'b']].agg(myprint2, 0)
   a  b  c  d
0  1  1  x  s
1  2  1  x  s
2  3  0  x  s
   a  b  c  d
3  4  1  z  d
4  5  1  z  d
5  6  0  z  d

For every group, I would have expected 2 calls of myprint2, one with Series 'a' and one with Series 'b'. This is particularly confusing, because agg would have the expected behaviour if groupby is called over a list or if a function of 1 argument is passed to agg:

Expected Output

I would have expected the same as if groupby is called over a list or if a function of 1 argument is passed to agg:

>>>> df.groupby(['c', 'd'])[['a', 'b']].agg(myprint2, 0)
0    1
1    2
2    3
Name: a, dtype: int64
3    4
4    5
5    6
Name: a, dtype: int64
0    1
1    1
2    0
Name: b, dtype: int64
3    1
4    1
5    0
Name: b, dtype: int64
>>>> df.groupby(['c'])[['a', 'b']].agg(print)
0    1
1    2
2    3
Name: a, dtype: int64
3    4
4    5
5    6
Name: a, dtype: int64
0    1
1    1
2    0
Name: b, dtype: int64
3    1
4    1
5    0
Name: b, dtype: int64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.None

pandas : 1.0.3
numpy : 1.18.2
pytz : 2019.3
dateutil : 2.8.1
pip : 20.0.2
setuptools : 46.1.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
pyxlsb : None
s3fs : None
scipy : 1.4.1
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : 1.2.0
xlwt : None
xlsxwriter : None
numba : None

[Dr-Irv]

This is quite subtle, as when you supply a function to agg, the extra argument for your function makes it so that a DataFrame is passed. Here's an example, using the above DataFrame

In [108]: def onearg(x):
     ...:     print('x is ', x)
     ...:     print('x dtype is', type(x))
     ...:     if isinstance(x, pd.DataFrame):
     ...:         print('numcols is ', len(x.columns))
     ...:     else:
     ...:         print("not a DataFrame")
     ...:     return x.sum()
     ...:
In [111]: def twoargs(x, y):
     ...:     print('x is ', x)
     ...:     print('x dtype is', type(x))
     ...:     if isinstance(x, pd.DataFrame):
     ...:         print('numcols is ', len(x.columns))
     ...:     else:
     ...:         print("not a DataFrame")
     ...:     return x.sum()
     ...:

The functions do the same thing, printing the argument, the type of the argument, and printing the number of columns if it is a DataFrame and then return the sum.

But the output is different:

In [109]: df.groupby('c')[['a', 'b']].agg(onearg)
x is  0    1
1    2
2    3
Name: a, dtype: int64
x dtype is <class 'pandas.core.series.Series'>
not a DataFrame
x is  3    4
4    5
5    6
Name: a, dtype: int64
x dtype is <class 'pandas.core.series.Series'>
not a DataFrame
x is  0    1
1    1
2    0
Name: b, dtype: int64
x dtype is <class 'pandas.core.series.Series'>
not a DataFrame
x is  3    1
4    1
5    0
Name: b, dtype: int64
x dtype is <class 'pandas.core.series.Series'>
not a DataFrame

In [112]: df.groupby('c')[['a', 'b']].agg(twoargs, 0)
x is     a  b  c  d
0  1  1  x  s
1  2  1  x  s
2  3  0  x  s
x dtype is <class 'pandas.core.frame.DataFrame'>
numcols is  4
x is     a  b  c  d
3  4  1  z  d
4  5  1  z  d
5  6  0  z  d
x dtype is <class 'pandas.core.frame.DataFrame'>
numcols is  4

Without the extra argument, a series was passed in to the function. With the extra argument, the entire DataFrame was passed in.

The behavior here is not documented. So it might be intentional, or possibly a bug.

[rhshadrach]

Closing as a duplicate of #39169