Skip to content

BUG: concat with copy=False of ExtensionArray fails #20756

Closed
@jorisvandenbossche

Description

@jorisvandenbossche
In [1]: from pandas.tests.extension.decimal.array import DecimalArray, make_data

In [5]: dec_arr = DecimalArray(make_data())

In [6]: df1 = pd.DataFrame({'int1': [1, 2, 3], 'key':[0, 1, 2], 'ext1': dec_arr[:3]})

In [7]: df2 = pd.DataFrame({'int2': [1, 2, 3, 4], 'key':[0, 0, 1, 3], 'ext2': dec_arr[3:7]})

In [8]: pd.concat([df1, df1], axis=1, copy=False)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-60-bd0d3639db5b> in <module>()
----> 1 pd.concat([df1, df1], axis=1, copy=False)

/home/joris/scipy/pandas/pandas/core/reshape/concat.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, copy)
    211                        verify_integrity=verify_integrity,
    212                        copy=copy)
--> 213     return op.get_result()
    214 
    215 

/home/joris/scipy/pandas/pandas/core/reshape/concat.py in get_result(self)
    406             new_data = concatenate_block_managers(
    407                 mgrs_indexers, self.new_axes, concat_axis=self.axis,
--> 408                 copy=self.copy)
    409             if not self.copy:
    410                 new_data._consolidate_inplace()

/home/joris/scipy/pandas/pandas/core/internals.py in concatenate_block_managers(mgrs_indexers, axes, concat_axis, copy)
   5394     import pdb; pdb.set_trace()
   5395 
-> 5396     for placement, join_units in concat_plan:
   5397 
   5398         if len(join_units) == 1 and not join_units[0].indexers:

AttributeError: 'DecimalArray' object has no attribute 'view'

this fails because of:

if len(join_units) == 1 and not join_units[0].indexers:
b = join_units[0].block
values = b.values
if copy:
values = values.copy()
elif not copy:
values = values.view()
b = b.make_block_same_class(values, placement=placement)

so if copy=False, it takes a view of the data, which is not defined on the extension array interface.
I am not fully sure why the view is needed here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ExtensionArrayExtending pandas with custom dtypes or arrays.Needs TestsUnit test(s) needed to prevent regressionsgood first issue

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions