Skip to content

AttributeError(s) if .loc[] is passed a list with missing labels on a non-unique index of Sparse object #20766

Closed
@toobaz

Description

@toobaz

Code Sample, a copy-pastable example if possible

In [2]: s = pd.Series(-1, index=list('aba')).to_sparse()

In [3]: s.loc[['a', 'c']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
Out[3]: ---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~/.local/lib/python3.5/site-packages/IPython/core/formatters.py in __call__(self, obj)
    700                 type_pprinters=self.type_printers,
    701                 deferred_pprinters=self.deferred_printers)
--> 702             printer.pretty(obj)
    703             printer.flush()
    704             return stream.getvalue()

~/.local/lib/python3.5/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    393                             if callable(meth):
    394                                 return meth(obj, self, cycle)
--> 395             return _default_pprint(obj, self, cycle)
    396         finally:
    397             self.end_group()

~/.local/lib/python3.5/site-packages/IPython/lib/pretty.py in _default_pprint(obj, p, cycle)
    508     if _safe_getattr(klass, '__repr__', None) is not object.__repr__:
    509         # A user-provided repr. Find newlines and replace them with p.break_()
--> 510         _repr_pprint(obj, p, cycle)
    511         return
    512     p.begin_group(1, '<')

~/.local/lib/python3.5/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    699     """A pprint that just redirects to the normal repr function."""
    700     # Find newlines and replace them with p.break_()
--> 701     output = repr(obj)
    702     for idx,output_line in enumerate(output.splitlines()):
    703         if idx:

/home/nobackup/repo/pandas/pandas/core/base.py in __repr__(self)
     80         Yields Bytestring in Py2, Unicode String in py3.
     81         """
---> 82         return str(self)
     83 
     84 

/home/nobackup/repo/pandas/pandas/core/base.py in __str__(self)
     59 
     60         if compat.PY3:
---> 61             return self.__unicode__()
     62         return self.__bytes__()
     63 

/home/nobackup/repo/pandas/pandas/core/sparse/series.py in __unicode__(self)
    261         series_rep = Series.__unicode__(self)
    262         rep = '{series}\n{index!r}'.format(series=series_rep,
--> 263                                            index=self.sp_index)
    264         return rep
    265 

/home/nobackup/repo/pandas/pandas/core/generic.py in __getattr__(self, name)
   4376             if name in self._info_axis:
   4377                 return self[name]
-> 4378             return object.__getattribute__(self, name)
   4379 
   4380     def __setattr__(self, name, value):

/home/nobackup/repo/pandas/pandas/core/sparse/series.py in sp_index(self)
    200     @property
    201     def sp_index(self):
--> 202         return self.block.sp_index
    203 
    204     @property

AttributeError: 'FloatBlock' object has no attribute 'sp_index'

In [4]: df = pd.DataFrame(-1, index=list('abc'), columns=list('xyx')).to_sparse()

In [5]: df.loc[:, ['x', 'w']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
/home/nobackup/repo/pandas/pandas/core/indexing.py:1368: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  return self._getitem_tuple(key)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-56a2111d1155> in <module>()
----> 1 df.loc[:, ['x', 'w']]

/home/nobackup/repo/pandas/pandas/core/indexing.py in __getitem__(self, key)
   1366             except (KeyError, IndexError):
   1367                 pass
-> 1368             return self._getitem_tuple(key)
   1369         else:
   1370             # we by definition only have the 0th axis

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_tuple(self, tup)
    875                 continue
    876 
--> 877             retval = getattr(retval, self.name)._getitem_axis(key, axis=i)
    878 
    879         return retval

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1829                     raise ValueError('Cannot index with multidimensional key')
   1830 
-> 1831                 return self._getitem_iterable(key, axis=axis)
   1832 
   1833             # nested tuple slicing

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_iterable(self, key, axis)
   1156                     result = result._reindex_with_indexers(
   1157                         {axis: [new_target, new_indexer]},
-> 1158                         copy=True, allow_dups=True)
   1159 
   1160                 else:

/home/nobackup/repo/pandas/pandas/core/sparse/frame.py in _reindex_with_indexers(self, reindexers, method, fill_value, limit, copy, allow_dups)
    731                                                 fill_value=fill_value)
    732             else:
--> 733                 new_arrays[col] = self[col]
    734 
    735         return self._constructor(new_arrays, index=index,

/home/nobackup/repo/pandas/pandas/core/sparse/frame.py in __getitem__(self, key)
    439             return self._getitem_array(key)
    440         else:
--> 441             return self._get_item_cache(key)
    442 
    443     def get_value(self, index, col, takeable=False):

/home/nobackup/repo/pandas/pandas/core/generic.py in _get_item_cache(self, item)
   2482         if res is None:
   2483             values = self._data.get(item)
-> 2484             res = self._box_item_values(item, values)
   2485             cache[item] = res
   2486             res._set_as_cached(item, self)

/home/nobackup/repo/pandas/pandas/core/frame.py in _box_item_values(self, key, values)
   3047         items = self.columns[self.columns.get_loc(key)]
   3048         if values.ndim == 2:
-> 3049             return self._constructor(values.T, columns=items, index=self.index)
   3050         else:
   3051             return self._box_col_values(values, items)

AttributeError: 'BlockManager' object has no attribute 'T'

Problem description

Despite the two different errors, I guess they are related.

Probably related to #14427

Might be also related to

Expected Output

In [6]: s.to_dense().loc[['a', 'c']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
Out[6]: 
a   -1.0
a   -1.0
c    NaN
dtype: float64

In [10]: df.to_dense().loc[:, ['x', 'w']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
/home/nobackup/repo/pandas/pandas/core/indexing.py:1368: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  return self._getitem_tuple(key)
Out[10]: 
   x  x   w
a -1 -1 NaN
b -1 -1 NaN
c -1 -1 NaN

Output of pd.show_versions()

INSTALLED VERSIONS

commit: d04b746
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-6-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: it_IT.UTF-8

pandas: 0.23.0.dev0+754.gd04b7464d
pytest: 3.5.0
pip: 9.0.1
setuptools: 39.0.1
Cython: 0.25.2
numpy: 1.14.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.5.6
patsy: 0.5.0
dateutil: 2.7.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0dev
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.3.0
xlsxwriter: 0.9.6
lxml: 4.1.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.2.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    IndexingRelated to indexing on series/frames, not to indexes themselvesSparseSparse Data Type

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions