Adding DataFrameImputer

[gsmafra]

Code adapted from http://stackoverflow.com/questions/25239958/impute-categorical-missing-values-in-scikit-learn

[dukebody]

Hi @gsmafra. First of all, thanks for your contribution!

I think having an imputer for a string column that imputes with the most frequent value is a cool feature. However, given that sklearn-pandas allows you to select to which columns apply which transformers, I believe it's better to have the "most frequent string value imputer" separated from the traditional mean/median imputer, already implemented in sklearn.preprocessing.Imputer.

Could you refactor your code into a MostFrequentValueImputer class that does the following?

• Takes a numpy array as input
• Outputs a numpy array
• For every column, imputes the NaNs with the most frequent value in that column

Thanks!

[gsmafra]

Hi @dukebody, thanks for your answer

Sure I can do that, but wouldn't it make more sense to add this functionality directly on scikit-learn if that is what we want to do?

Also, sklearn.preprocessing.Imputer already has a most_frequent option, but it doesn't accept strings in the input, so the best name would probably be StringImputer or CategoricalImputer, don't you agree?

[dukebody]

This is implicitly assuming that there will be only one mode value. Can you raise an explicit exception if this is not true? Something like:

modes = pd.Series(X).mode()
if modes.shape[0] == 0:
    raise ValueError('No value is repeteated more than twice in the column')
elif modes.shape[0] > 1:
    raise ValueError('Column has multiple modes {}, can't select one to fill'.format(modes.tolist())
else:
    self.fill = modes[0]

[dukebody]

I merged a rebase of your work here in b40328c

Thanks for your contribution! And sorry for the delay.

[dukebody]

If you want you can submit another PR to improve what I mentioned in the comment.