Skip to content

Allow drop bins when using the cut function #20947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 10, 2018
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -526,6 +526,7 @@ Other Enhancements
- Added new writer for exporting Stata dta files in version 117, ``StataWriter117``. This format supports exporting strings with lengths up to 2,000,000 characters (:issue:`16450`)
- :func:`to_hdf` and :func:`read_hdf` now accept an ``errors`` keyword argument to control encoding error handling (:issue:`20835`)
- :func:`date_range` now returns a linearly spaced ``DatetimeIndex`` if ``start``, ``stop``, and ``periods`` are specified, but ``freq`` is not. (:issue:`20808`)
- :func:`cut` has gained the ``duplicates='raise'|'drop'`` option to control whether to raise on duplicated edges (:issue:`20947`)

.. _whatsnew_0230.api_breaking:

Expand Down
38 changes: 35 additions & 3 deletions pandas/core/reshape/tile.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@


def cut(x, bins, right=True, labels=None, retbins=False, precision=3,
include_lowest=False):
include_lowest=False, duplicates='raise'):
"""
Bin values into discrete intervals.

Expand Down Expand Up @@ -65,6 +65,10 @@ def cut(x, bins, right=True, labels=None, retbins=False, precision=3,
The precision at which to store and display the bins labels.
include_lowest : bool, default False
Whether the first interval should be left-inclusive or not.
duplicates : {default 'raise', 'drop'}, optional
If bin edges are not unique, raise ValueError or drop non-uniques.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a versionadded tag

add a Raises part of the doc-string


.. versionadded:: 0.23.0

Returns
-------
Expand All @@ -85,7 +89,8 @@ def cut(x, bins, right=True, labels=None, retbins=False, precision=3,
bins : numpy.ndarray or IntervalIndex.
The computed or specified bins. Only returned when `retbins=True`.
For scalar or sequence `bins`, this is an ndarray with the computed
bins. For an IntervalIndex `bins`, this is equal to `bins`.
bins. If set `duplicates=drop`, `bins` will drop non-unique bin. For
an IntervalIndex `bins`, this is equal to `bins`.

See Also
--------
Expand Down Expand Up @@ -144,6 +149,32 @@ def cut(x, bins, right=True, labels=None, retbins=False, precision=3,
dtype: category
Categories (3, interval[float64]): [(1.992, 4.667] < (4.667, ...

Passing a Series as an input returns a Series with mapping value.
It is used to map numerically to intervals based on bins.

>>> s = pd.Series(np.array([2, 4, 6, 8, 10]),
... index=['a', 'b', 'c', 'd', 'e'])
>>> pd.cut(s, [0, 2, 4, 6, 8, 10], labels=False, retbins=True, right=False)
... # doctest: +ELLIPSIS
(a 0.0
b 1.0
c 2.0
d 3.0
e 4.0
dtype: float64, array([0, 2, 4, 6, 8]))

Use `drop` optional when bins is not unique

>>> pd.cut(s, [0, 2, 4, 6, 10, 10], labels=False, retbins=True,
... right=False, duplicates='drop')
... # doctest: +ELLIPSIS
(a 0.0
b 1.0
c 2.0
d 3.0
e 3.0
dtype: float64, array([0, 2, 4, 6, 8]))

Passing an IntervalIndex for `bins` results in those categories exactly.
Notice that values not covered by the IntervalIndex are set to NaN. 0
is to the left of the first bin (which is closed on the right), and 1.5
Expand Down Expand Up @@ -199,7 +230,8 @@ def cut(x, bins, right=True, labels=None, retbins=False, precision=3,
fac, bins = _bins_to_cuts(x, bins, right=right, labels=labels,
precision=precision,
include_lowest=include_lowest,
dtype=dtype)
dtype=dtype,
duplicates=duplicates)

return _postprocess_for_cut(fac, bins, retbins, x_is_series,
series_index, name)
Expand Down
16 changes: 16 additions & 0 deletions pandas/tests/reshape/test_tile.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import numpy as np
from pandas.compat import zip

import pandas as pd
from pandas import (DataFrame, Series, isna, to_datetime, DatetimeIndex, Index,
Timestamp, Interval, IntervalIndex, Categorical,
cut, qcut, date_range, NaT, TimedeltaIndex)
Expand Down Expand Up @@ -337,6 +338,21 @@ def test_series_retbins(self):
CDT(ordered=True))
tm.assert_series_equal(result, expected)

def test_cut_duplicates_bin(self):
# issue 20947
values = Series(np.array([1, 3, 5, 7, 9]),
index=["a", "b", "c", "d", "e"])
bins = [0, 2, 4, 6, 10, 10]
result = cut(values, bins, duplicates='drop')
expected = cut(values, pd.unique(bins))
tm.assert_series_equal(result, expected)

pytest.raises(ValueError, cut, values, bins)
pytest.raises(ValueError, cut, values, bins, duplicates='raise')

# invalid
pytest.raises(ValueError, cut, values, bins, duplicates='foo')

def test_qcut_duplicates_bin(self):
# GH 7751
values = [0, 0, 0, 0, 1, 2, 3]
Expand Down