ENH: Add duplicate="ignore" to pd.cut

[exandi]

Research

• I have searched the [pandas] tag on StackOverflow for similar questions.

• I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

https://stackoverflow.com/questions/71671922/pandas-qcut-function-duplicates-parameter

Question about pandas

I Am new to this, so I will copy paste from Stackoverflow. I maybe could also provide a pull-request. But maybe I oversaw something?

Maybe I don't get the point? but why isn't in Pandas qcut function accepting "ignore" as argument from duplicates?

So small Datasets with duplicate Values are printing the Error: "Bin edges must be unique" and the advice to use the "drop" option. But if you want to have a fixed Number of bins there is no possibility?

small code example thats not working:

import pandas as pd
import numpy as np

data=pd.Series([1,1,2,3])
pd.qcut(data,10,labels=np.arange(0,10),duplicates="raise")

small code how it works, but don't get the same number of bins:

import pandas as pd
import numpy as np

data=pd.Series([1,1,2,3])
qcut(data,4,labels=np.arange(0,3),duplicates="drop")

What could be a possible solution:

• Insert a third option "ignore" to

  pandas/pandas/core/reshape/tile.py

  Line 405 in 06d2301

  if duplicates not in ["raise", "drop"]:

• Change the if else block in

  pandas/pandas/core/reshape/tile.py

  Lines 418 to 424 in 06d2301

  	if duplicates == "raise":
  	raise ValueError(
  	f"Bin edges must be unique: {repr(bins)}.\n"
  	f"You can drop duplicate edges by setting the 'duplicates' kwarg"
  	)
  	else:
  	bins = unique_bins

  to

if duplicates == "raise":
   raise ValueError(
       f"Bin edges must be unique: {repr(bins)}.\n"
       f"You can drop duplicate edges by setting the 'duplicates' kwarg"
   )
elif duplicates == "drop":
   bins = unique_bins

[mroeschke]

If you want a fixed number of bins you can just use cut? https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.cut.html

[exandi]

The functionalaty of cut and qcut is different.
with data as:

data=pd.Series([1,1,2,3])

the qcut returns the following:

pd.qcut(data,4,labels=np.arange(0,3),duplicates="drop")

0 0
1 0
2 1
3 2

the qcut returns the fowllowing:

print(pd.cut(data,4,labels=np.arange(0,4),duplicates="drop"))

0 0
1 0
2 1
3 3

if the distribution of the data is bigger so the range of min and max raises liks:

data=pd.Series([1,1,2,100])

the output of qcut stays the same

pd.qcut(data,4,labels=np.arange(0,3),duplicates="drop")

0 0
1 0
2 1
3 2

the output of cut changes, because it consider the distances between each values.

pd.cut(data,4,labels=np.arange(0,4),duplicates="drop")

0 0
1 0
2 0
3 3

cut considers the datarange and qcut considers the amount of a single value.

hope my question is now more understanable? I will be on vacation for the next 3 weeks so my next replay will take some time

[mroeschke]

From the original post, it's not clear to me whether you're asking for duplicate="ignore" or are reporting a bug regarding the duplicate parameter

[exandi]

I am asking for duplicate="ignore"