Skip to content

PERF/ENH: np.apply_along_axis -> reduce in moments.py/nanops.py #3185

Closed
@jreback

Description

@jreback

http://stackoverflow.com/questions/15652343/how-to-speed-up-pandas-rolling-sum
http://stackoverflow.com/questions/21058333/compute-rolling-maximum-drawdown-of-pandas-series

this numpy recipe might be of iterest to create a strided dimension:

numpy/numpy#31

prob easiest in these cases to create a algos.roll_generic_2d

./pandas/core/frame.py: (this is OK, just a fallback if reduce doesn't work)
 4109 :             result = np.apply_along_axis(func, axis, self.values)
./pandas/core/nanops.py:
  201 :         return np.apply_along_axis(get_median, axis, values)
  235 :             result = np.apply_along_axis(__builtin__.min, apply_ax, values)
  260 :             result = np.apply_along_axis(__builtin__.max, apply_ax, values)
./pandas/core/panel.py:
 1246 :         result = np.apply_along_axis(func, i, self.values)
./pandas/stats/moments.py:
  281 :         result = np.apply_along_axis(calc, axis, values)
  363 :     output = np.apply_along_axis(_ewma, 0, values)
  651 :     result = np.apply_along_axis(f, axis, values)
   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1000003    8.930    0.000    8.930    0.000 {method 'put' of 'numpy.ndarray' objects}
     1000    8.413    0.008   23.225    0.023 shape_base.py:11(apply_along_axis)
  1000000    3.612    0.000    3.612    0.000 {pandas.algos.roll_sum}
  1000000    0.833    0.000    5.277    0.000 moments.py:277(<lambda>)
  1000000    0.628    0.000    4.444    0.000 moments.py:499(call_cython)
  2000000    0.551    0.000    0.551    0.000 {method 'tolist' of 'numpy.ndarray' objects}
        2    0.517    0.259    0.517    0.259 {pandas.algos.take_2d_axis0_float64_float64}

Metadata

Metadata

Assignees

No one assigned

    Labels

    GroupbyPerformanceMemory or execution speed performanceReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions