ENH: DataFrame.to_csv support for "compression='gzip'"

[francescomalandrino]

the DataFrame.to_csv method seems to accept a "compression" named parameter:

import numpy as np,pandas as pd
data=np.arange(10).reshape(5,2)
df=pd.DataFrame(data,columns=['a','b'])
df.to_csv('test.csv.gz',compression='gzip')

However, the file it creates is not compressed at all:
francesco@i3 ~/Desktop $ cat test.csv.gz
,a,b
0,0,1
1,2,3
2,4,5
3,6,7
4,8,9

How about either (i) actually implementing compression, or at least (ii) raise an error? The current behavior is confusing...

[jreback]

to_csv allows **kwds so arbitrary additional arguments are 'accepted' (this is mainly for compatibility IIRC with some of the other to_* functions which allow this), but ignored. I suppose that could be removed (not sure why it was their in the first place). That said, only arguments in the doc-string are public.

Would accept a pull-request to limit this.

[mmautner]

I think this out-of-scope for Pandas--just use this: https://docs.python.org/2/library/gzip.html

please close

[francescomalandrino]

But compression='gzip' is accepted (and enacted) in pd.read_csv, which is why I was assuming to_csv behaves the same.

[mmautner]

The way you initially phrased the issue suggested that you were just guessing at keyword arguments--'compression' isn't a documented argument so I don't think your confusion is shared by many. You're welcome to submit a pull-request, I don't feel religious about this at all

[francescomalandrino]

Sorry, what I meant is:

1. compression is documented and working for read_csv:
   http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.parsers.read_csv.html
2. compression is not documented and not working for to_csv.

There is no error in the documentation and both (1) and (2) make sense to me.
It's (1) and (2) together, i.e., the fact that to_csv behaves differently from read_csv without telling the user, that seemed a bit inconsistent to me.

Closing on the grounds that I won't be fixing it myself, and probably it's not a proper bug.

[mmautner]

Thanks! I definitely didn't mean to antagonize you--agreed that it's an unfortunate inconsistency

[jorisvandenbossche]

Would we want this feature, if someone would implement it? If so, we can leave it open marked as an enhancement proposal?