read_fwf does not take into account skiprows when inferring column width

[xchardon]

read_fwf can automatically detect column widths, but that does not seem to take into account rows to be skipped (parameter skiprows).
So, if the file contains text before the data, column detection fails. It could be solved by using only the actual data lines (and header) to infer column widths.
I worked around that issue by opening the text file, calling readline as many times as necessary (= skiprows), and then calling read_fwf on that buffer (with skiprows=0)

[jreback]

can you post a copy-pastable example which repros this?

[xchardon]

Here is a minimal example:

import pandas as pd

with open('testfwf.txt', 'w') as f:
    f.write("Text contained in the file header\n")
    f.write("\n")
    f.write("DataCol1   DataCol2\n")
    f.write("     0.0        1.0\n")
    f.write("   101.6      956.1\n")

skiprows = 2

print("Using the read_fwf function simply:")
df = pd.read_fwf('testfwf.txt', skiprows = skiprows)
print(list(df))

print("Manually skipping the text lines before using read_fwf:")
with open('testfwf.txt', 'r') as f:
    for i in range(skiprows):
        f.readline()
    df = pd.read_fwf(f, encoding="utf8")
    print(list(df))

It gives the following result:

Using the read_fwf function simply:
['DataCol1   DataCol2', 'Unnamed: 1', 'Unnamed: 2']
Manually skipping the text lines before using read_fwf:
['DataCol1', 'DataCol2']

I'm using Python 3.4.3 and pandas 0.16.2

[jreback]

ok, looks like a bug. pull-requests are welcome to fix!