BUG: unstack with missing levels results in incorrect index names #38029
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
arw2019
suggested changes
Nov 24, 2020
doc/source/whatsnew/v1.2.0.rst
Outdated
| @@ -619,6 +619,7 @@ Indexing | |||
| - Bug in indexing on a :class:`Series` or :class:`DataFrame` with a :class:`CategoricalIndex` using listlike indexer that contains elements that are in the index's ``categories`` but not in the index itself failing to raise ``KeyError`` (:issue:`37901`) | |||
| - Bug in :meth:`DataFrame.iloc` and :meth:`Series.iloc` aligning objects in ``__setitem__`` (:issue:`22046`) | |||
| - Bug in :meth:`DataFrame.loc` did not raise ``KeyError`` when missing combination was given with ``slice(None)`` for remaining levels (:issue:`19556`) | |||
| - Bug in :meth:`MultiIndex.remove_unused_levels` drops NaN when level contains NaN (:issue:`37510`) | |||
There was a problem hiding this comment.
... was dropping missing values when levels contain ``NaN``
but also do we want to mention something about set_levels since that was what the OP was about
| df1.index = df1.index.set_levels(levels=new_levels, level="id1") | ||
| df1.index = df1.index.set_levels(levels=new_levels, level="id2") | ||
|
|
||
| result = df1.unstack("id3")[("x", 1)].sort_index().index |
There was a problem hiding this comment.
I'd say construct the expected frame and compare using assert_frame_equal
alternatively use tm.assert_index_equal
| # GH 37510 | ||
| df1 = DataFrame( | ||
| { | ||
| "id1": [1, 2, 3, 4], |
There was a problem hiding this comment.
total nit but I'd call these L1', L2, L3`
| @@ -271,3 +271,24 @@ def test_argsort(idx): | |||
| result = idx.argsort() | |||
| expected = idx.values.argsort() | |||
| tm.assert_numpy_array_equal(result, expected) | |||
|
|
|||
|
|
|||
| def test_not_remove_nan(): | |||
There was a problem hiding this comment.
I'd do test_remove_unused_levels_with_missing
jreback
requested changes
Nov 26, 2020
GYHHAHA
changed the title
|
frame hardcoded, cc @jreback |
jreback
requested changes
Dec 29, 2020
doc/source/whatsnew/v1.2.0.rst
Outdated
| @@ -755,6 +755,7 @@ Reshaping | |||
| - Bug in :meth:`DataFrame.apply` not setting index of return value when ``func`` return type is ``dict`` (:issue:`37544`) | |||
| - Bug in :func:`concat` resulting in a ``ValueError`` when at least one of both inputs had a non-unique index (:issue:`36263`) | |||
| - Bug in :meth:`DataFrame.merge` and :meth:`pandas.merge` returning inconsistent ordering in result for ``how=right`` and ``how=left`` (:issue:`35382`) | |||
| - Bug in :meth:`DataFrame.unstack` with missing levels led to incorrect index names (:issue:`37510`) | |||
jreback
requested changes
Dec 31, 2020
doc/source/whatsnew/v1.2.0.rst
Outdated
| @@ -619,6 +619,7 @@ Indexing | |||
| - Bug in indexing on a :class:`Series` or :class:`DataFrame` with a :class:`CategoricalIndex` using listlike indexer that contains elements that are in the index's ``categories`` but not in the index itself failing to raise ``KeyError`` (:issue:`37901`) | |||
| - Bug in :meth:`DataFrame.iloc` and :meth:`Series.iloc` aligning objects in ``__setitem__`` (:issue:`22046`) | |||
| - Bug in :meth:`DataFrame.loc` did not raise ``KeyError`` when missing combination was given with ``slice(None)`` for remaining levels (:issue:`19556`) | |||
| - Bug in :meth:`MultiIndex.remove_unused_levels` drops NaN when level contains NaN (:issue:`37510`) | |||
jreback
approved these changes
Dec 31, 2020
|
thanks @GYHHAHA |
luckyvs1
pushed a commit
to luckyvs1/pandas
that referenced
this pull request
Jan 20, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff