API: update pd.cut with the new categorical integration

[jorisvandenbossche]

xref #8074. Some issues about pd.cut/qcut (http://pandas.pydata.org/pandas-docs/stable/generated/pandas.cut.html#pandas.cut):

• If the input is a Series, also return a Series?
• The produced categorical is not ordered, while the Categorical constructor is by default. Also in the case of cut it seems logical that it would be ordered?
• The docstring of cut makes use of labels, but I think here the individual categories are meant, and not the numerical representation (codes) (still another use of labels, good we changed that name! :-)), but I am not fully sure of the explanation:

  • "Labels to use for bin edges" -> why the 'edges'? Are it not just the labels for the bins itself? (this sets what we now call the 'levels'):

    In [23]: pd.cut([1,2,3,4], bins=3, labels=['a', 'b', 'c'])
    Out[23]:
    a
    a
    b
    c
    Levels (3, object): [a, b, c]
    

  • "or False to return integer bin labels" -> this is what now is called 'codes'?

  • should we rename this to levels/categories?

[jankatins]

• cut produces an unordered cat: That it isn't ordered is simple an oversight on my part when I changed the cut to the new fastpath constructor and forgot to specify ordered.
• re labels: labels as a name is used at quite a lot places in pandas and I haven't really tried to understand why they are called 'labels'. @jreback can probably say more about this...
• re produce a Series: IMO it would be nice to return a Series, if the input was a Series and a Categorical in all else cases ("Categorical" is usually used like a np.array)

[jreback]

@JanSchulz I think the 'labels' comes from R, @wesm originally put this in place.

[jorisvandenbossche]

From the R docs (http://stat.ethz.ch/R-manual/R-devel/library/base/html/cut.html):

labels : labels for the levels of the resulting category. By default, labels are constructed using "(a,b]" interval notation. If labels = FALSE, simple integer codes are returned instead of a factor.

[jreback]

R is very intuitive!

[jankatins]

Ah, ok, then in this case labels are actually used as "description for something" and my misunderstandings come from different places, where it is an array of ints used in indexing and what was cat.labels in the old Categorical :-)

R's factor has two inputs: levels and labels. Levels are first used to construct levels from the data and labels are then used to rename the levels (so after applying labels, you don't have any references to the original level names).

factor(vals, levels=levels, labels=labels) is translated into current pandas as

cat = pd.Categorical(val, levels=levels)
cat.levels = labels

So in this cases I think it makes sense to use either levels or (if this is renamed) categories.

[jorisvandenbossche]

yep, based on that information, I also think the labels as is now in the cut docstring should in the pandas naming scheme rather be levels/categories

[jankatins]

Do we have an agreement here that the result of pd.cut/pd.dcut should be a Series if the input was a Series?

[jreback]

yes that makes sense

[jorisvandenbossche]

+1

[jankatins]

Just to make it clear: should cut also now get a categories and not labels argument or should we keep the labels as an argument and just make it clear in the docstring that "If you supply label strings for the bins, these will end up the categories."?

I really dread to rename all labels into categories throughout the pandas codebase...