Groupby Pct_change Bug Report

[AnakinShieh]

Groupby Pct_change Bug Report

from io import StringIO
import pandas as pd

txt = StringIO("""Group;Date;Value
  A     ;01-02-2016     ;16 
  A     ;01-03-2016     ;15 
  A     ;01-04-2016     ;14 
  A     ;01-05-2016     ;17 
  A     ;01-06-2016     ;19 
  A     ;01-07-2016     ;20 
  B     ;01-02-2016     ;16 
  B     ;01-03-2016     ;13 
  B     ;01-04-2016     ;13 
  C     ;01-02-2016     ;16 
  C     ;01-03-2016     ;16 """)

df = pd.read_table(txt,sep=';',parse_dates=[1])
d1 = df.set_index(['Date', 'Group']).Value
d2 = d1.groupby(level='Group').pct_change()
print(d2)

Problem description

I tried to calculate value changes by group in pandas , according this stackoverflow answer, https://stackoverflow.com/questions/41453325/python-pandas-groupby-calculate-change

the expected result should be

Expected Output

Date        Group
2016-01-02  A             NaN
2016-01-03  A       -0.062500
2016-01-04  A       -0.066667
2016-01-05  A        0.214286
2016-01-06  A        0.117647
2016-01-07  A        0.052632
2016-01-02  B             NaN
2016-01-03  B       -0.187500
2016-01-04  B        0.000000
2016-01-02  C             NaN
2016-01-03  C        0.000000
Name: Value, dtype: float64

First pecent change value of each group should be NaN, while in my computer with pandas version 0.23.1, the result shows as below.

Date        Group   
2016-01-02    A              NaN
2016-01-03    A        -0.062500
2016-01-04    A        -0.066667
2016-01-05    A         0.214286
2016-01-06    A         0.117647
2016-01-07    A         0.052632
2016-01-02    B        -0.200000
2016-01-03    B        -0.187500
2016-01-04    B         0.000000
2016-01-02    C         0.230769
2016-01-03    C         0.000000
Name: Value, dtype: float64

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.5.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 23 Model 1 Stepping 1, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.1
pytest: 3.5.1
pip: 10.0.1
setuptools: 39.1.0
Cython: 0.28.2
numpy: 1.14.3
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 6.4.0
sphinx: 1.7.4
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.4
blosc: None
bottleneck: 1.2.1
tables: 3.4.3
numexpr: 2.6.5
feather: None
matplotlib: 2.2.2
openpyxl: 2.5.3
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.4
lxml: 4.2.1
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.2.7
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[WillAyd]

Thanks for the report - investigation and PRs are always welcome!

[mroeschke]

Is this a dupe of #21200?

[WillAyd]

Not exactly a duplicate but will probably make sense to fix in the same PR. Not opposed to closing this and managing just through that if you feel that is cleaner @mroeschke

[mroeschke]

I see. IMO I'd probably be better to merge these issues (especially if you think the fix is the same). Looks like PR #21235 is open to fix #21200, I'll inquire if there's bandwidth to cover this issue as well in that PR.

[simonariddell]

Sorry for the late reply @mroeschke . However, when I run your example code on my branch for PR #21235 I do get the expected output, so this will solve the issue.