ENH: specificy missing labels in loc calls GH34272 #34912

timhunderwood · 2020-06-20T18:48:43Z

closes ENH: specify missing keys in KeyError when passing list-like to .loc #34272
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Added what's new documentation.

…h-specificy-missing-labels-34272

Flake 8 line length linting.

timhunderwood · 2020-06-20T18:53:22Z

This should address #34272.

Would it be useful to add max limit to the number of missing labels it would print? I think this will be handled better by the str method of the Series class, already.

Fix ipython directive to code block so that error message can be included.

jreback · 2020-06-20T20:25:30Z

doc/source/whatsnew/v1.1.0.rst

+.. code-block:: ipython
+
+    s = pd.Series({"a": 1, "b": 2, "c": 3})
+    s.loc[["a", "b", "missing_0", "c", "missing_1", "missing_2"]]


you don’t need a section for this just a note in the whats new

sure, I've removed the code block and just left the text entry in the what's new.

jreback · 2020-06-20T20:26:36Z

pandas/core/indexing.py

@@ -1302,10 +1303,12 @@ def _validate_read_indexer(
            # code, so we want to avoid warning & then
            # just raising
            if not ax.is_categorical():
+                not_found = list(key[missing_mask])


instead of doing this you can create an Index which has the natural formatters

Done, and I now also use the option context to limit the number of items displayed.

Note, the if raise_missing block just above this uses a list to print the missing keys here. Independently of this PR, if raise_missing==False, an error would be raised directly below regardless (unless the axis is categorical). Not sure if this is the expected/desired behaviour? Perhaps one for another ticket/PR.

jreback · 2020-06-20T20:27:43Z

pandas/tests/indexing/test_indexing.py

@@ -1075,3 +1075,12 @@ def test_setitem_with_bool_mask_and_values_matching_n_trues_in_length():
    result = ser
    expected = pd.Series([None] * 3 + list(range(5)) + [None] * 2).astype("object")
    tm.assert_series_equal(result, expected)
+
+
+def test_missing_labels_inside_loc():


can u also try with a lot of missing labels (testing that message is constrained)

you will need to set the max line width via option context

Good point, done. I added a test for many labels and a test for very long labels.

Note because we are raising a KeyError the message is wrapped in quotes (see stack overflow and https://bugs.python.org/issue2651 . This means the \n characters appear in the message.

The pre-existing error message text here is also quite long as it includes a link to the docs and text on "no longer supported". Let me know if you think it should be shortened.

Fixed rst flake8 issues.

pandas/tests/indexing/test_indexing.py

PR feedback: what's new code block removed; additional test for many or long missing labels, context manager for display options.

jreback

small comments ping on green

jreback · 2020-06-25T01:32:16Z

pandas/core/indexing.py

-                    "https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike"  # noqa:E501
-                )
+                not_found = key[missing_mask]
+                from pandas import option_context


import at the top

jreback · 2020-06-25T01:35:16Z

doc/source/whatsnew/v1.1.0.rst

+Previously, if labels were missing for a loc call, a Key Error was raised stating that this was no longer supported.
+
+Now the error message also includes a list of the missing labels (max 10 items, display width 80 characters).
+


can u add the issue number here

can you do it like other issue references, e.g. :issue:`34272`

@jreback - sure, this is now done.

PR feedback: reference issue in what's new and import position.

PR feedback: import sorting (isort) fix

jreback

ping when green

jreback · 2020-06-25T22:14:39Z

doc/source/whatsnew/v1.1.0.rst

+Previously, if labels were missing for a loc call, a Key Error was raised stating that this was no longer supported.
+
+Now the error message also includes a list of the missing labels (max 10 items, display width 80 characters).
+


can you do it like other issue references, e.g. :issue:`34272`

PR feedback: add issue tag

PR feedback: minor edit of what is new message.

jreback · 2020-06-26T20:21:22Z

thanks @timhunderwood very nice

timhunderwood · 2020-06-26T20:23:40Z

thanks, Happy to help!

timhunderwood added 4 commits June 20, 2020 19:21

ENH: Include missing labels in error message for loc GH34272

63fbce4

ENH: Include missing labels in error message for loc GH34272

140794b

Added what's new documentation.

Merge branch 'master' of https://github.com/pandas-dev/pandas into en…

cde8c59

…h-specificy-missing-labels-34272

ENH: Include missing labels in error message for loc GH34272

ac23b02

Flake 8 line length linting.

ENH: Include missing labels in error message for loc GH34272

6b1ad07

Fix ipython directive to code block so that error message can be included.

jreback requested changes Jun 20, 2020

View reviewed changes

jreback added Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves labels Jun 20, 2020

ENH: Include missing labels in error message for loc GH34272

8710596

Fixed rst flake8 issues.

mroeschke reviewed Jun 21, 2020

View reviewed changes

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

ENH: Include missing labels in error message for loc GH34272

b77dcfc

PR feedback: what's new code block removed; additional test for many or long missing labels, context manager for display options.

jreback requested changes Jun 25, 2020

View reviewed changes

timhunderwood added 2 commits June 25, 2020 20:43

ENH: Include missing labels in error message for loc GH34272

c460a03

PR feedback: reference issue in what's new and import position.

ENH: Include missing labels in error message for loc GH34272

ecaf345

PR feedback: import sorting (isort) fix

jreback added this to the 1.1 milestone Jun 25, 2020

jreback requested changes Jun 25, 2020

View reviewed changes

timhunderwood added 2 commits June 25, 2020 23:22

ENH: Include missing labels in error message for loc GH34272

744c828

PR feedback: add issue tag

ENH: Include missing labels in error message for loc GH34272

18d2b83

PR feedback: minor edit of what is new message.

jreback approved these changes Jun 26, 2020

View reviewed changes

jreback merged commit deec940 into pandas-dev:master Jun 26, 2020

fangchenli pushed a commit to fangchenli/pandas that referenced this pull request Jun 27, 2020

ENH: specificy missing labels in loc calls GH34272 (pandas-dev#34912)

c0470a7

		Previously, if labels were missing for a loc call, a Key Error was raised stating that this was no longer supported.

		Now the error message also includes a list of the missing labels (max 10 items, display width 80 characters).

Uh oh!

ENH: specificy missing labels in loc calls GH34272 #34912

ENH: specificy missing labels in loc calls GH34272 #34912

Uh oh!

Conversation

timhunderwood commented Jun 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timhunderwood commented Jun 20, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timhunderwood Jun 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jreback left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timhunderwood Jun 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jreback left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jreback commented Jun 26, 2020

Uh oh!

timhunderwood commented Jun 26, 2020

Uh oh!

Uh oh!

timhunderwood commented Jun 20, 2020 •

edited

Loading

timhunderwood Jun 21, 2020 •

edited

Loading

timhunderwood Jun 26, 2020 •

edited

Loading