Skip to content

BUG: Preserve sparse dtype when reindexing #26183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

alex-hutton
Copy link

@alex-hutton alex-hutton commented Apr 22, 2019

Hi, I've been taking a look at this but I need to verify some of the surrounding functionality so that I prevent introducing more breakages.

In this test, https://github.com/alex-hutton/pandas/blob/preserve-sparse-dtype-26123/pandas/tests/sparse/test_combine_concat.py#L99-L110 , my interpretation is that its purpose is to verify that if two SparseSeries are concatenated, a fill_value set on either of those SparseSeries should be ignored in favour of the default fill_value of NaN.

Is my interpretation correct, and is this test correct? It seems odd that the the fill_value of the SparseSeries should be disregarded, but I am probably missing something.

@pep8speaks
Copy link

pep8speaks commented Apr 22, 2019

Hello @alex-hutton! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-04-22 12:35:45 UTC

@alex-hutton alex-hutton force-pushed the preserve-sparse-dtype-26123 branch from 534aed6 to b8754b6 Compare April 22, 2019 12:34
 - This is currently broken due to many tests
   relying on possibly broken behaviour.
 - BUG pandas-dev#26123.
@alex-hutton alex-hutton force-pushed the preserve-sparse-dtype-26123 branch from b8754b6 to 6b9035c Compare April 22, 2019 12:35
@alex-hutton alex-hutton changed the title Preserve sparse dtype when reindexing BUG: Preserve sparse dtype when reindexing Apr 22, 2019
@gfyoung gfyoung added Bug Regression Functionality that used to work in a prior pandas version Sparse Sparse Data Type and removed Bug labels Apr 22, 2019
@gfyoung gfyoung requested a review from jreback April 22, 2019 21:55
@@ -4527,7 +4527,8 @@ def _reindex_with_indexers(self, reindexers, fill_value=None, copy=False,
if copy and new_data is self._data:
new_data = new_data.copy()

return self._constructor(new_data).__finalize__(self)
kwargs = {'dtype': self._data.dtype} if preserve_dtype else {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens if you always just pass the dtype of the original in

@jreback
Copy link
Contributor

jreback commented Jun 8, 2019

closing as stale. if you'd to continue, pls merge master and ping.

@jreback jreback closed this Jun 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Regression Functionality that used to work in a prior pandas version Sparse Sparse Data Type
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reindexing a sparse data structure with a different index results in losing the dtype
4 participants