ENH: pd.Series.shift and .diff to accept a collection of numbers

[jona-sassenhagen]

Is your feature request related to a problem?

No

Describe the solution you'd like

pd.DataFrame.shift/pd.Series.shift could alternatively accept a list of ints or a collection of numbers, and the result would be a dataframe with one column per shift. This would make it very convenient to e.g. create multiple lagged versions of a column for time series problems.

E.g.,

time_series_df[my_time_feature].shift([-3, -2, -1, 0, 1, 2, 3])

would give me a dataframe with 7 columns, each shifted versions of time_series_df[my_time_feature].

API breaking implications

The first input to df.shift would be overloaded to also accept a list of ints.

Describe alternatives you've considered

Alternatives are very simple to do, you just iterate over the lags and concatenate or join.
This would just be a handy shortcut.

Additional context

I'd in principle be open to implementing this myself, although I've never contributed to pandas.

[rhshadrach]

+1 - This is pretty easy to accomplish today, but pandas accepts lists in many places like this and it feels like a natural extension of that. A few thoughts if we do go forward with this:

• Should accept any iterable
• Should raise a TypeError if any value is not an integer
• A list of one element should return a DataFrame and not a Series
• Need to handle the case of an empty list (raise a ValueError?)

[rhshadrach]

Would DataFrame.shift also need to accept an iterable?

[jona-sassenhagen]

I would say for the sake of consistency both series and dataframe should accept the iterable.

This raises the question of column order for the Dataframe case.

[jreback]

this should just be a series operation only or if ina frame would have to generate a multi index

[jona-sassenhagen]

this should just be a series operation only or if ina frame would have to generate a multi index

You mean, have either lags or columns on the index?
This would be annoying for the use case of analysing time series data with e.g. a regression, where you want one column per lag and feature. My idea would be to default to having the list of ints case be the same as a concatenation of looping over ints.

[rhshadrach]

For Series, what would the columns be labeled? Is there another Series method that can take multiple inputs and act as a transformer? I am only aware of reducers (e.g. quantile).

[jona-sassenhagen]

Is there another Series method that can take multiple inputs and act as a transformer?

pd.Series.transform right? It returns a series if called with a single function and a df if called with a list of functions.

For Series, what would the columns be labeled?

I would label them as some function of the shift and the original name. E.g. orig_column_shift_0, orig_column_shift_1 etc.

[jbrockmendel]

xref #19298

[GitChein]

take