API/DOC: status of low_memory kwarg of read_csv/table

[cancan101]

I am getting the following warning:

/usr/local/lib/python2.7/dist-packages/pandas-0.13.0rc1_78_g142ca62-py2.7-linux-x86_64.egg/pandas/io/parsers.py:1050: 
DtypeWarning: Columns (6) have mixed types. Specify dtype option on import or set low_memory=False.
  data = self._reader.read(nrows)

but I can find no documentation for low_memory

[jreback]

its a kind of deprecated option (but still works)

[cancan101]

If the low_memory parameter is deprecated, it should be marked as such. Also the warning message should be removed or re-worded.

[jreback]

I said kind of deprecated in that I don't think it's necessary anymore

[randyzwitch]

As another data point, I got this warning about mixed types. Setting to low_memory=False as suggested actually crashed Python (Win7 64-bit, through IPython Notebook). I'm nowhere near my memory limits, so removing the argument and keeping the warning was no big deal for me.

[jorisvandenbossche]

There is also an example in the book "Python for Data Analysis" that leads to this warning (p278):

In [13]: fec = pd.read_csv('ch09/P00000001-ALL.csv')

C:\Anaconda\lib\site-packages\pandas\io\parsers.py:1130: DtypeWarning: Columns (6) have mixed types. Specify dtype option on import or set low_memory=False.
  data = self._reader.read(nrows)

So I agree with above:

• or the option is deprecated, and then the mention in the warning should be removed
• or the option is not deprecated and should be documented (and should not crash your python)

From the code, it does not seem like deprecated (it is still used: https://github.com/pydata/pandas/blob/master/pandas/parser.pyx#L727), but it seems that it is given a default value of True in read_csv regardless of what you specify (https://github.com/pydata/pandas/blob/master/pandas/io/parsers.py#L354). There are also still tests specifically for high memery: https://github.com/pydata/pandas/blob/master/pandas/io/tests/test_parsers.py#L2897