Segfault on clean-up with count example from docstrings

[jorisvandenbossche]

In #19952 I have problems with a segfault when running doctests, and I could isolate at least the following code snippet that segfaults on clean-up (so only when exiting the process; an explicit del + gc.collect does not yet trigger the segfault).

import numpy as  np
import pandas as pd

df = pd.DataFrame({"Person": ["John", "Myla", None, "John", "Myla"],
                   "Age": [24., np.nan, 21., 33, 26],
                   "Single": [False, True, True, True, False]})
                  
res = df.set_index(["Person", "Single"]).count(level="Person")

[jorisvandenbossche]

OK, and the cause is the missing value in the 'Person' level.

Simplified example:

In [1]: df = pd.DataFrame({"A": ["a", "b", None, "a", "b"],
   ...:                    "B": 1,
   ...:                    "C": range(5)})
   ...:                   
   ...: res = df.set_index(["A", "B"]).count(level="A")

In [2]: res
Out[2]: 
   B
A   
a  2
b  2

is causing the segfault (and without the None it does not).
But, when selecting the column, it does not produce a segfault, but it also gives a different result (the NaN is included in index):

In [1]: df = pd.DataFrame({"A": ["a", "b", None, "a", "b"],
   ...:                    "B": 1,
   ...:                    "C": range(5)})
   ...:                   
   ...: res = df.set_index(["A", "B"])['C'].count(level="A")

In [2]: res
Out[2]: 
A
a      2
b      2
NaN    1
Name: C, dtype: int64

Here the segfault does not happen, but it seems buggy behaviour, as count should not count the NaNs ?

[jreback]

is this example fixed? is this just a tracking issue for finding the cause?

@pandas-dev/pandas-core

[gfyoung]

@jreback : Not really sure? The examples don't seem to segfault for me...

cc @jorisvandenbossche

Actually, I take that back. They do. Just as I exited the Python console.