Skip to content

rolling_cov and rolling_corr error out when len(series) < min_periods #7764

Closed
@creeson

Description

@creeson

I am updating a codebase from an OLD version of pandas (0.7.3) to current version (0.14.1) in python 2.7.

pandas: 0.14.1
numpy: 1.8.0

Not sure if this is properly classified as a bug, or if it is an expected code change. However, it seems rolling_cov / rolling_corr are specifying the window as min(window given, max(len(arg1), len(arg2)), and then erring out if arg1 AND arg2 are shorter than min_periods. Before, we would just get NaNs back.

Here is some test code:

import numpy as np
import pandas as pd

np.random.seed(0)

s1 = pd.Series(np.random.randn(4))
s2 = pd.Series(np.random.randn(4))

print '\nrolling_corr:'
try:
    print pd.rolling_corr(s1, s2, window=10, min_periods=5)
except Exception as inst:
    print type(inst)
    print inst
print '\nrolling_cov:'
try:
    print pd.rolling_cov(s1, s2, window=10, min_periods=5)
except Exception as inst:
    print type(inst)
    print inst

In 0.14.1:

rolling_corr:
<type 'exceptions.ValueError'>
min_periods (5) must be <= window (4)
rolling_cov:
<type 'exceptions.ValueError'>
min_periods (5) must be <= window (4)

In 0.7.3:

rolling_corr:
0   NaN
1   NaN
2   NaN
3   NaN

rolling_cov:
0   NaN
1   NaN
2   NaN
3   NaN

Note that changing either s1 OR s2 to be 6, the code doesn't error out, and instead outputs:

rolling_corr:
0   NaN
1   NaN
2   NaN
3   NaN
4   NaN
5   NaN
dtype: float64

rolling_cov:
0   NaN
1   NaN
2   NaN
3   NaN
4   NaN
5   NaN
dtype: float64

Meanwhile, in 0.14.1, other rolling functions continue to behave as before:

s1 = pd.Series(np.random.randn(4))

print '\rolling_std:'
print pd.rolling_std(s1, window=10, min_periods=5)
print '\nrolling_max:'
print pd.rolling_max(s1, window=10, min_periods=5)

Outputs:

rolling_std:
0   NaN
1   NaN
2   NaN
3   NaN
dtype: float64

rolling_max:
0   NaN
1   NaN
2   NaN
3   NaN
dtype: float64

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugError ReportingIncorrect or improved errors from pandasNumeric OperationsArithmetic, Comparison, and Logical operations

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions