Skip to content

pandas.api.types.infer_dtype should be nan aware #17059

Closed
@cpcloud

Description

@cpcloud

Code Sample, a copy-pastable example if possible

# Your code here
In [6]: s = pd.Series(['a', np.nan, 'c'])

In [7]: pd.api.types.infer_dtype(s)
Out[7]: 'mixed'

Problem description

infer_dtype gives mixed as the inferred dtype when all values are of the same type modulo NaNs. I think we should change this to be NaN aware so that downstream tools can take advantage of this inference when operating on pandas objects.

Expected Output

Ideally,

In [8]: pd.api.types.infer_dtype(s.dropna())
Out[8]: 'string'

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.11.6-3-ARCH
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: en_US.UTF-8

pandas: 0.20.2
pytest: 3.0.7
pip: 9.0.1
setuptools: 27.2.0
Cython: None
numpy: 1.13.1
scipy: 0.19.1
xarray: None
IPython: 6.0.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.9999999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: 2.9.5
s3fs: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

Labels

EnhancementMissing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions