Skip to content

PERF/ENH: np.apply_along_axis -> reduce in moments.py/nanops.py #3185

@jreback

Description

@jreback

http://stackoverflow.com/questions/15652343/how-to-speed-up-pandas-rolling-sum
http://stackoverflow.com/questions/21058333/compute-rolling-maximum-drawdown-of-pandas-series

this numpy recipe might be of iterest to create a strided dimension:

numpy/numpy#31

prob easiest in these cases to create a algos.roll_generic_2d

./pandas/core/frame.py: (this is OK, just a fallback if reduce doesn't work) 4109 : result = np.apply_along_axis(func, axis, self.values) ./pandas/core/nanops.py: 201 : return np.apply_along_axis(get_median, axis, values) 235 : result = np.apply_along_axis(__builtin__.min, apply_ax, values) 260 : result = np.apply_along_axis(__builtin__.max, apply_ax, values) ./pandas/core/panel.py: 1246 : result = np.apply_along_axis(func, i, self.values) ./pandas/stats/moments.py: 281 : result = np.apply_along_axis(calc, axis, values) 363 : output = np.apply_along_axis(_ewma, 0, values) 651 : result = np.apply_along_axis(f, axis, values) 
 Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1000003 8.930 0.000 8.930 0.000 {method 'put' of 'numpy.ndarray' objects} 1000 8.413 0.008 23.225 0.023 shape_base.py:11(apply_along_axis) 1000000 3.612 0.000 3.612 0.000 {pandas.algos.roll_sum} 1000000 0.833 0.000 5.277 0.000 moments.py:277(<lambda>) 1000000 0.628 0.000 4.444 0.000 moments.py:499(call_cython) 2000000 0.551 0.000 0.551 0.000 {method 'tolist' of 'numpy.ndarray' objects} 2 0.517 0.259 0.517 0.259 {pandas.algos.take_2d_axis0_float64_float64} 

Metadata

Metadata

Assignees

No one assigned

    Labels

    GroupbyPerformanceMemory or execution speed performanceReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions