Closed
Description
http://stackoverflow.com/questions/16023584/pandas-time-delta-from-grouped-neighbors
setup
txt = '''ID,DATE
002691c9cec109e64558848f1358ac16,2003-08-13 00:00:00
002691c9cec109e64558848f1358ac16,2003-08-13 00:00:00
0088f218a1f00e0fe1b94919dc68ec33,2006-05-07 00:00:00
0088f218a1f00e0fe1b94919dc68ec33,2006-06-03 00:00:00
00d34668025906d55ae2e529615f530a,2006-03-09 00:00:00
00d34668025906d55ae2e529615f530a,2006-03-09 00:00:00
0101d3286dfbd58642a7527ecbddb92e,2007-10-13 00:00:00
0101d3286dfbd58642a7527ecbddb92e,2007-10-27 00:00:00
0103bd73af66e5a44f7867c0bb2203cc,2001-02-01 00:00:00
0103bd73af66e5a44f7867c0bb2203cc,2008-01-20 00:00:00
'''
df = pandas.read_csv(StringIO.StringIO(txt))
df = df.sort('DATE')
df.DATE = pandas.to_datetime(df.DATE)
grouped = df.groupby('A')
-
this should probably work? (fill in a 0 timedelta)
grouped.apply(lambda g: g['DATE']-g['DATE'].shift()).fillna(0)
-
this is returning
timedelta64[us]
, weird numpy error again
In [34]: grouped.apply(lambda g: g['DATE']-g['DATE'].shift())
Out[34]:
8 NaT
0 NaT
1 00:00:00
4 NaT
5 00:00:00
2 NaT
3 27 days, 00:00:00
6 NaT
7 14 days, 00:00:00
9 2544 days, 00:00:00
Name: DATE, dtype: timedelta64[us]
This DOES work
In [57]: df['X_SEQUENCE_GAP'].sort_index().astype('timedelta64[ns]').fillna(0)
Out[57]:
0 00:00:00
1 00:00:00
2 00:00:00
3 27 days, 00:00:00
4 00:00:00
5 00:00:00
6 00:00:00
7 14 days, 00:00:00
8 00:00:00
9 2544 days, 00:00:00
Name: X_SEQUENCE_GAP, dtype: timedelta64[ns]
This DOES NOT WORK, (need to use a view of i8 first)
In [58]: df['X_SEQUENCE_GAP'].sort_index().astype('timedelta64[ns]').ffill()
ValueError: Invalid dtype for pad_1d [timedelta64[ns]]