Skip to content

BUG: DecimalArray and JSONArray that are empty return incorrect results for isna() #21189

Closed
@Dr-Irv

Description

@Dr-Irv

Code Sample, a copy-pastable example if possible

In [1]: import pandas as pd

In [2]: from pandas.tests.extension.decimal.array import DecimalArray

In [3]: from pandas.tests.extension.json.array import JSONArray

In [4]: DecimalArray([]).isna()
Out[4]: array([], dtype=float64)

In [5]: JSONArray([]).isna()
Out[5]: array([], dtype=float64)

In [6]: import decimal

In [7]: DecimalArray([decimal.Decimal(1.0)]).isna()
Out[7]: array([False])

In [8]: DecimalArray([decimal.Decimal(1.0)]).isna().dtype
Out[8]: dtype('bool')

Problem description

This relates to a discussion with @TomAugspurger in #21183 when I made (since redacted) changes to pandas/util/testing.py in assert_extension_array_equal . It turns out that the implementations of isna() in DecimalArray and JSONArray return an ndarray of the wrong dtype if the arrays are empty.

So we need to force the dtype of the result of isna() to be bool in those implementations

Expected Output

In [4]: DecimalArray([]).isna()
Out[4]: array([], dtype=bool)

In [5]: JSONArray([]).isna()
Out[5]: array([], dtype=bool)

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.0
pytest: 3.3.2
pip: 9.0.1
setuptools: 38.4.0
Cython: 0.27.3
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.6.6
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: None
matplotlib: 2.1.2
openpyxl: 2.4.10
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.1
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: 1.2.1
pymysql: 0.7.11.None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugExtensionArrayExtending pandas with custom dtypes or arrays.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions