The itertools
module in Python provides a collection of fast, memory-efficient tools for performing iterators. These functions work on iterables and produce iterators as output. The module is a standard library, so no installation is required.
Table of Contents
- Introduction
- Infinite Iterators
count
cycle
repeat
- Iterators Terminating on the Shortest Input Sequence
accumulate
chain
compress
dropwhile
filterfalse
groupby
islice
starmap
takewhile
tee
zip_longest
- Combinatoric Iterators
product
permutations
combinations
combinations_with_replacement
- Examples
- Basic Usage of Infinite Iterators
- Using
chain
to Flatten a List of Lists - Generating Permutations and Combinations
- Real-World Use Case
- Conclusion
- References
Introduction
The itertools
module provides a set of functions for creating and using iterators. These iterators can be used to perform various operations such as concatenation, grouping, filtering, and more.
Infinite Iterators
count
Returns an iterator that generates consecutive integers, starting from the provided start
value and incremented by step
.
import itertools counter = itertools.count(start=10, step=2) for _ in range(5): print(next(counter))
Output:
10 12 14 16 18
cycle
Returns an iterator that cycles through the elements in the provided iterable indefinitely.
import itertools cycler = itertools.cycle('ABCD') for _ in range(8): print(next(cycler))
Output:
A B C D A B C D
repeat
Returns an iterator that repeats the provided value indefinitely or up to a specified number of times.
import itertools repeater = itertools.repeat('Hello', 3) for item in repeater: print(item)
Output:
Hello Hello Hello
Iterators Terminating on the Shortest Input Sequence
accumulate
Returns an iterator that generates accumulated sums, or accumulated results of other binary functions provided via the func
argument.
import itertools data = [1, 2, 3, 4, 5] accumulated = itertools.accumulate(data) print(list(accumulated))
Output:
[1, 3, 6, 10, 15]
chain
Returns an iterator that generates elements from the first iterable until it is exhausted, then from the next iterable, and so on.
import itertools chain = itertools.chain([1, 2, 3], ['a', 'b', 'c']) print(list(chain))
Output:
[1, 2, 3, 'a', 'b', 'c']
compress
Returns an iterator that filters elements from the first iterable, only returning those that have a corresponding element in selectors that evaluates to True
.
import itertools data = ['a', 'b', 'c', 'd'] selectors = [1, 0, 1, 0] compressed = itertools.compress(data, selectors) print(list(compressed))
Output:
['a', 'c']
dropwhile
Returns an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element.
import itertools data = [1, 2, 3, 4, 5] dropped = itertools.dropwhile(lambda x: x < 3, data) print(list(dropped))
Output:
[3, 4, 5]
filterfalse
Returns an iterator that filters elements from the iterable, only returning those for which the predicate is false.
import itertools data = [1, 2, 3, 4, 5] filtered = itertools.filterfalse(lambda x: x % 2, data) print(list(filtered))
Output:
[2, 4]
groupby
Returns an iterator that generates consecutive keys and groups from the iterable. The key is a function computing a key value for each element.
import itertools data = sorted([('a', 1), ('b', 2), ('a', 3), ('b', 4)]) grouped = itertools.groupby(data, key=lambda x: x[0]) for key, group in grouped: print(key, list(group))
Output:
a [('a', 1), ('a', 3)] b [('b', 2), ('b', 4)]
islice
Returns an iterator that generates selected elements from the iterable. If start
is specified, it skips all preceding elements; step
determines the interval between elements.
import itertools data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] sliced = itertools.islice(data, 2, 8, 2) print(list(sliced))
Output:
[2, 4, 6]
starmap
Returns an iterator that computes the function using arguments obtained from the iterable.
import itertools import operator data = [(2, 3), (4, 5), (6, 7)] starmapped = itertools.starmap(operator.mul, data) print(list(starmapped))
Output:
[6, 20, 42]
takewhile
Returns an iterator that returns elements from the iterable as long as the predicate is true.
import itertools data = [1, 2, 3, 4, 5] taken = itertools.takewhile(lambda x: x < 4, data) print(list(taken))
Output:
[1, 2, 3]
tee
Returns n
independent iterators from a single iterable.
import itertools data = [1, 2, 3, 4, 5] iters = itertools.tee(data, 2) for it in iters: print(list(it))
Output:
[1, 2, 3, 4, 5] [1, 2, 3, 4, 5]
zip_longest
Returns an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue
.
import itertools data1 = [1, 2, 3] data2 = ['a', 'b'] zipped = itertools.zip_longest(data1, data2, fillvalue='X') print(list(zipped))
Output:
[(1, 'a'), (2, 'b'), (3, 'X')]
Combinatoric Iterators
product
Returns the Cartesian product of input iterables.
import itertools prod = itertools.product([1, 2], ['a', 'b']) print(list(prod))
Output:
[(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]
permutations
Returns successive permutations of elements in the iterable.
import itertools perm = itertools.permutations([1, 2, 3]) print(list(perm))
Output:
[(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)]
combinations
Returns successive combinations of elements in the iterable.
import itertools comb = itertools.combinations([1, 2, 3], 2) print(list(comb))
Output:
[(1, 2), (1, 3), (2, 3)]
combinations_with_replacement
Returns successive combinations of elements in the iterable, allowing individual elements to be repeated more than once.
import itertools comb_wr = itertools.combinations_with_replacement([1, 2, 3], 2) print(list(comb_wr))
Output:
[(1, 1), (1, 2), (1, 3), (2, 2), (2, 3), (3, 3)]
Examples
Basic Usage of Infinite Iterators
import itertools counter = itertools.count(start=5, step=5) for _ in range(5): print(next(counter)) cycler = itertools.cycle('AB') for _ in range(4): print(next(cycler)) repeater = itertools.repeat('Hello', 3) for item in repeater: print(item)
Output:
5 10 15 20 25 A B A B Hello Hello Hello
Using chain to Flatten a List of Lists
import itertools lists = [[1, 2, 3], [4, 5], [6, 7, 8]] flattened = itertools.chain(*lists) print(list(flattened))
Output:
[1, 2, 3, 4, 5, 6, 7, 8]
Generating Permutations and Combinations
import itertools items = ['A', 'B', 'C'] perm = itertools.permutations(items) print("Permutations:", list(perm)) comb = itertools.combinations(items, 2) print("Combinations:", list(comb)) comb_wr = itertools.combinations_with_replacement(items, 2) print("Combinations with Replacement:", list(comb_wr))
Output:
Permutations: [('A', 'B', 'C'), ('A', 'C', 'B'), ('B', 'A', 'C'), ('B', 'C', 'A'), ('C', 'A', 'B'), ('C', 'B', 'A')] Combinations: [('A', 'B'), ('A', 'C'), ('B', 'C')] Combinations with Replacement: [('A', 'A'), ('A', 'B'), ('A', 'C'), ('B', 'B'), ('B', 'C'), ('C', 'C')]
Real-World Use Case
Generating Passwords
Generate all possible passwords using a given set of characters and a fixed length.
import itertools characters = 'abc123' length = 3 passwords = itertools.product(characters, repeat=length) for password in passwords: print(''.join(password))
Output:
aaa aab aac aa1 aa2 aa3 aba abb abc ab1 ab2 ab3 aca acb acc ac1 ac2 ac3 a1a a1b a1c a11 a12 a13 a2a a2b a2c a21 a22 a23 a3a a3b a3c a31 a32 a33 baa bab bac ba1 ba2 ba3 bba bbb bbc bb1 bb2 bb3 bca bcb bcc bc1 bc2 bc3 b1a b1b b1c b11 b12 b13 b2a b2b b2c b21 b22 b23 b3a b3b b3c b31 b32 b33 caa cab cac ca1 ca2 ca3 cba cbb cbc cb1 cb2 cb3 cca ccb ccc cc1 cc2 cc3 c1a c1b c1c c11 c12 c13 c2a c2b c2c c21 c22 c23 c3a c3b c3c c31 c32 c33 1aa 1ab 1ac 1a1 1a2 1a3 1ba 1bb 1bc 1b1 1b2 1b3 1ca 1cb 1cc 1c1 1c2 1c3 11a 11b 11c 111 112 113 12a 12b 12c 121 122 123 13a 13b 13c 131 132 133 2aa 2ab 2ac 2a1 2a2 2a3 2ba 2bb 2bc 2b1 2b2 2b3 2ca 2cb 2cc 2c1 2c2 2c3 21a 21b 21c 211 212 213 22a 22b 22c 221 222 223 23a 23b 23c 231 232 233 3aa 3ab 3ac 3a1 3a2 3a3 3ba 3bb 3bc 3b1 3b2 3b3 3ca 3cb 3cc 3c1 3c2 3c3 31a 31b 31c 311 312 313 32a 32b 32c 321 322 323 33a 33b 33c 331 332 333
Grouping Data
Group students by their grades.
import itertools students = [('John', 'A'), ('Jane', 'B'), ('Dave', 'A'), ('Sue', 'B'), ('Tim', 'C')] students.sort(key=lambda x: x[1]) grouped = itertools.groupby(students, key=lambda x: x[1]) for grade, group in grouped: print(f"Grade {grade}: {[student[0] for student in group]}")
Output:
Grade A: ['John', 'Dave'] Grade B: ['Jane', 'Sue'] Grade C: ['Tim']
Conclusion
The itertools
module in Python provides a powerful set of functions for creating and manipulating iterators. These functions can be used to perform a variety of operations efficiently and with minimal memory usage.