Skip to content

read_csv: if parse_dates dont appear in use_cols, we get a trace #31251

Closed
@teto

Description

@teto

Code Sample, a copy-pastable example if possible

# Your code here
import pandas as pd
import io

content = io.StringIO('''
time,val
212.23, 32
''')

date_cols = ['time']

df = pd.read_csv(
    content,
    sep=',',
    usecols=['val'],
    dtype= { 'val': int },
    parse_dates=date_cols,
)

Problem description

triggers

Traceback (most recent call last):
  File "test.py", line 16, in <module>
    parse_dates=date_cols,
  File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py",
line 685, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py",
line 463, in _read
    data = parser.read(nrows)
  File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py",
line 1154, in read
    ret = self._engine.read(nrows)
  File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py",
line 2134, in read
    names, data = self._do_date_conversions(names, data)
  File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py",
line 1885, in _do_date_conversions
    keep_date_col=self.keep_date_col,
  File "/nix/store/k4fd48jzsyafvcifa6wi6pk4vaprnw36-python3.7-pandas-0.25.3/lib/python3.7/site-packages/pandas/io/parsers.py",
line 3335, in _process_date_conversion
    data_dict[colspec] = converter(data_dict[colspec])
KeyError: 'time'

i.e., if you use columns in parse_dates that dont appear in use_cols, then you are screwed.

Either parse_dates sould be added to use_cols or the documentation precise it. An assert could make the error more understandable too.
pandas 0.25.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions