ENH: Add skipna to groupby.first and groupby.last

[hsorsky]

Edit[rhshadrach]: This bug report has been reworked into an enhancement request. See the discussion below.

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd

X = pd.DataFrame(
    {
        "group": [1, 1],
        "value": [np.nan, 1],
    },
    index=[42, 0]
)

X.groupby(["group"]).transform("nth", n=0)

Issue Description

DataFrameGroupBy.transform no longer supports "nth", n=0 as an argument.

I assume this is related to the general push to limit what is allowed to be passed to transform to only operations that return a single row per group, and the fact that "nth" can return multiple rows if n is a list or such.

Unfortunately, "first" is not an acceptable alternative because it removes NaNs, which might not be desired. The current work around is something like lambda g: g.iloc[0] but this forces us to use UDFs which are slow.

---

I apologise if this has already been discussed, I tried to look for related content but couldn't find anything. If it is, I would be happy to turn this into a feature request for something like an update to first to allow it to not filter out NaNs.

Expected Behavior

The above code would return the 1st row from each group, not ignoring NaNs, i.e.

    value
42    NaN
0     NaN

Installed Versions

INSTALLED VERSIONS
------------------
commit           : 2e218d10984e9919f0296931d92ea851c6a6faf5
python           : 3.10.11.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 22.4.0
Version          : Darwin Kernel Version 22.4.0: Mon Mar  6 20:59:28 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T6000
machine          : arm64
processor        : arm
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : en_US.UTF-8
pandas           : 1.5.3
numpy            : 1.26.3
pytz             : 2022.4
dateutil         : 2.8.2
setuptools       : 67.7.2
pip              : 23.2.1
Cython           : None
pytest           : 7.1.3
hypothesis       : 6.91.0
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : 4.9.2
html5lib         : 1.1
pymysql          : None
psycopg2         : 2.9.3
jinja2           : 3.1.2
IPython          : 8.5.0
pandas_datareader: None
bs4              : 4.11.1
bottleneck       : None
brotli           : 1.0.9
fastparquet      : None
fsspec           : 2022.8.2
gcsfs            : None
matplotlib       : 3.5.1
numba            : 0.58.1
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : 8.0.0
pyreadstat       : None
pyxlsb           : None
s3fs             : None
scipy            : 1.11.4
snappy           : None
sqlalchemy       : 1.4.41
tables           : None
tabulate         : 0.8.10
xarray           : None
xlrd             : None
xlwt             : None
zstandard        : None
tzdata           : None

[rhshadrach]

Thanks for the report. This was changed in 2.0, and the impact of not being able to transform with the first element group was not intentional. However, with nth now acting as a filter, groupby(...).transform("nth") would not produce the desired result even if we allowed it.

It seems to me that skipna=[[True]|False] in both first and last would make sense here, the default is backwards compatible, and consistent with other skipna defaults (I believe they all default to True). +1 on adding.

[hsorsky]

OK, makes sense. Would you like me to:

1. edit this issue and turn it into a feature request
2. create a new issue that is a feature request
3. something else (?)
4. do nothing

?

[rhshadrach]

@hsorsky - this looks to be in good shape now.