The attached program uncovers a two-fold performance regression in Python 2.5 with respect to Python 2.4. Below, the "element-wise" case goes from 2.5 seconds in 2.4 to about 4 in 2.5. Since that's a pretty serious increase, I thought I'd point it out. $ python2.5 periodic_test.py elwise 3.91989398003 collective 1.21577095985 $ python2.4 periodic_test.py elwise 2.50014710426 tuplewise 1.35589385033

I've narrowed the problem to the usage of generator expressions. Generator expressions appear to be MUCH slower on 2.5 than on 2.4. >python -m timeit "tuple([1 for _ in xrange(3)])" 2.4 -> 2.23us 2.5 -> 2.31us (a bit slower, but not so much) >python -m timeit "tuple(1 for _ in xrange(3))" 2.4 -> 3.32us 2.5 -> 8.03us (240% slower than 2.4) It appears to be a fixed cost, or startup cost, of the generator expression; differences get smaller when building large tuples (but always Python 2.5 runs slower than 2.4)

May this be a byproduct of the new generator features in 2.5?

The slowdown is due to the new function _PyObject_LengthHint: on my Win2K machine, the command python -m timeit "tuple(1 for _ in xrange(3))" answers: 100000 loops, best of 3: 4.14 usec per loop By adding a line "return rv;" on the second line of _PyObject_LengthHint, I get: 100000 loops, best of 3: 2.71 usec per loop Should we disable this function for some known built-in types?

I'll take a look at it next week -- want to think through the various cases. The current setup provides nice speedups when the length_hint is available. Possibly, it may be worthwhile to skip that check when the input is a generator. For the most part, I'm more concerned about the inner-loop time than the constant startup time outside the loop.

FWIW, the 2.4 to 2.5 timing difference came from renaming __len__ to __length_hint__. This was at Guido's request so that the value of bool (iter(obj)) would always be True. The consequence of the change was that we lost the fast slot lookup for __len__ and instead have to do a dictionary based attribute lookup. For the most part, I don't care about the overhead as it is constant. The inner-loop cost dominates. If you do care, the choices are to add some ugly, hackish specific type checks to bypass the attribute lookup in cases like generator objects where we know the type is absent. A more general, cleaner solution is to add a new slot for a length hint. Personally, I would rather leave this as is and live with the small constant lookup time. If you concur, please close this report. If not, please submit a patch for adding a new slot (model the code after that in PyObject_Size()).