DOC: Error in pd.cut documentation example regarding IntervalIndex usage.

[msznajder]

Code Sample, a copy-pastable example if possible

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.cut.html

bins = pd.IntervalIndex.from_tuples([(0, 1), (2, 3), (4, 5)])
pd.cut([0, 0.5, 1.5, 2.5, 4.5], bins)
[NaN, (0, 1], NaN, (2, 3], (4, 5]]
Categories (3, interval[int64]): [(0, 1] < (2, 3] < (4, 5]]

Problem description

Proposed example in pd.cut IntervalIndex section does not take into consideration actual pd.cut behaviour which in above example results produces ranges with lots of missing values and nans in results. In docs example above for example intermediate values like 2 and 4 WILL NOT be included in any bins, so the actual values of 2 and 4 in the data will produce nans after cutting the attribute using pd.cut.

I assume here that user in 99% of the time when using cut to bucketize value space wants all values in the spectrum to be included. This usage example can lead to data loss.

Actual example should be along the lines:

bins = pd.IntervalIndex.from_tuples([(0, 1), (1, 3), (3, 5)])

resulting in the following bins:

Categories (3, interval[int64]): [(0, 1] < (1, 3] < (3, 5]]

EDIT:
There is a correct/proper example in IntervalIndex docs:

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.IntervalIndex.from_tuples.html#pandas.IntervalIndex.from_tuples

[WillAyd]

I think the intent of that section is to show the NA value behavior right?

[msznajder]

Yeah I get it. What I mean is that docs should maybe present how to manually build the continuous example (90% of usages) using IntervalIndex instead of showing edge case of leaving out some values.

But you're right here that its not error as I flagged it - I did not recognized that this is section about missing values since I was looking for example of not missing them and got this one.

If my point of view is just mine the issue could be closed.

[WillAyd]

No worries. We typically always welcome improvements to the docs so if you see a way to make this better you can certainly push a PR

[jorisvandenbossche]

I think a more logical use case would be to show a value outside of the bin range to become NaN. That way you can have both: showing how a NaNs get created if the value is not in one of the bins, but still show a typical example where the manually created bins cover a full range without gaps.

@msznajder would you like to do a PR to update the example in that way?

[msznajder]

Sure thing - just give me time until weekend. I will push PR.

Not sure about the issue - should I close it now or after PR is pushed or accepted?

[WillAyd]

After PR is merged it will close