Description
Code Sample, a copy-pastable example if possible
df = pd.DataFrame({'a':[0,1,0,3], 'b':[2,0,0,5]})
sparse_df = df.to_sparse(fill_value=0)
criteria = (sparse_df['a'] > 2)
sparse_df_gt_2 = sparse_df.loc[criteria, :].copy()
sparse_df_lt_2 = sparse_df.loc[~criteria, :].copy() # this doesn't work
# simple problem solution:
sparse_df_lt_2 = sparse_df.loc[(criteria == False), :].copy()Problem description
When I want to do loc selection pandas dataframe with negated pandas SparseSeries it just doesn't work. I know how to solve this problem in another way, but I think there might be something wrong in the code.
This is an error message:
AssertionError Traceback (most recent call last)
in ()
3 criteria = (sparse_df['a'] > 2)
4 sparse_df_gt_2 = sparse_df.loc[criteria, :].copy()
----> 5 sparse_df_lt_2 = sparse_df.loc[~criteria, :].copy()
~/Projects/ml_projects/ml_python_venv/lib/python3.6/site-packages/pandas/core/generic.py in invert(self)
1142 try:
1143 arr = operator.inv(com._values_from_object(self))
-> 1144 return self.array_wrap(arr)
1145 except Exception:
1146
~/Projects/ml_projects/ml_python_venv/lib/python3.6/site-packages/pandas/core/sparse/series.py in array_wrap(self, result, context)
281 sparse_index=self.sp_index,
282 fill_value=fill_value,
--> 283 copy=False).finalize(self)
284
285 def array_finalize(self, obj):
~/Projects/ml_projects/ml_python_venv/lib/python3.6/site-packages/pandas/core/sparse/series.py in init(self, data, index, sparse_index, kind, fill_value, name, dtype, copy, fastpath)
114 data, sparse_index, fill_value = res
115 else:
--> 116 assert (len(data) == sparse_index.npoints)
117
118 elif isinstance(data, SingleBlockManager):
AssertionError:
Expected Output
Output of pd.show_versions()
INSTALLED VERSIONS
commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.15.0-34-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.23.4
pytest: None
pip: 18.0
setuptools: 39.1.0
Cython: None
numpy: 1.14.5
scipy: 1.1.0
pyarrow: 0.10.0
xarray: None
IPython: 6.5.0
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: 3.4.4
numexpr: 2.6.8
feather: None
matplotlib: 2.2.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: 2.7.5 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: 0.1.6
fastparquet: None
pandas_gbq: None
pandas_datareader: None