Skip to content

nrows incompatible with chunksize in read_csv #15755

Closed
@toobaz

Description

@toobaz

Code Sample, a copy-pastable example if possible

In [2]: pd.read_csv('/tmp/boh.csv', nrows=10, chunksize=2)
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-2-c652c48282ad> in <module>()
----> 1 pd.read_csv('/tmp/boh.csv', nrows=10, chunksize=2)

/home/nobackup/repo/pandas/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision)
    651                     skip_blank_lines=skip_blank_lines)
    652 
--> 653         return _read(filepath_or_buffer, kwds)
    654 
    655     parser_f.__name__ = name

/home/nobackup/repo/pandas/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    391 
    392     if (nrows is not None) and (chunksize is not None):
--> 393         raise NotImplementedError("'nrows' and 'chunksize' cannot be used"
    394                                   " together yet.")
    395     elif nrows is not None:

NotImplementedError: 'nrows' and 'chunksize' cannot be used together yet.

(as introduced by #6774 )

Problem description

Supporting them together is not complicated - PR on its way.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions