DEV Community

Super Kai (Kazuya Ito)
Super Kai (Kazuya Ito)

Posted on • Edited on

Iterator in Python (1)

#python #iterator #iter #iterable

Buy Me a Coffee☕

*Memo:

An iterator:

  • is the collection of zero or more elements which can return an element one by one:
    • Whether it's ordered or unordered, what type of an iterator it's, whether duplicated elements it can have and how many mixed types it can have all depend on the base type of an iterator.
  • can be huge with low memory usage so it doesn't get MemoryError.
  • is immutable so it cannot be changed.
  • has __iter__() and __next__().
  • can be iterated with a for statement.
  • can be unpacked with an assignment and for statement, function and * but not with **.
  • isn't False even if it's empty.
  • can be checked if a specific element is or isn't in it with in keyword or not and in keyword respectively.
  • can be checked if it is or isn't referred to by two variables with is keyword or not and is keyword respectively.
  • cannot be enlarged with * and a number.
  • can be created by iter() or __iter__() with an iterable, a generator, generator comprehension or itertools:
    • For iter(), the words type conversion are also suitable in addition to the word creation.
  • can be read by next(), __next__() or for loop to get each element one by one from the iterator.
  • cannot be read or changed by indexing or slicing.
  • raises StopIteration: if there are no elements to return.
  • can be continuously used through multiple variables.
  • except the one created by a generator or generator comprehension can be copied to refer to the same iterator.
  • can be used with len() after using list(), tuple(), set(), frozenset() or more_itertools.ilen() to get the length:
  • cannot be directly used with len() to get the length.

An iterator is for huge data so it doesn't get MemoryError.

iter() or __iter__() can create an iterator, then next(), __next__() or for loop can get each element one by one from the iterator as shown below:

*Memo:

  • iter():
    • The 1st argument is object(Required-Type:Iterable or Callable):
      • It's for an iterable if sentinel isn't set.
      • It's for a callable if sentinel is set.
      • Don't use object=.
    • The 2nd argument is sentinel(Optional-Type:Any):
      • It terminates object(Callable) if object(Callable) returns the same value as it:
        • The returned same value as it cannot be seen.
      • Don't use sentinel=.
  • __iter__() has no arguments.
  • next():
    • The 1st argument is iterator(Required-Type:Iterator):
      • Don't use iterator=.
    • The 2nd argument is default(Optional-Type:Any):
      • It's returned if the iterator is terminated:
        • Error occurs if it's not set and if the iterator is terminated.
      • Don't use default=.
  • __next__() has no arguments. 
v = iter([]) # Empty 1D iterator v = iter([0, 1, 2, 3, 4]) # 1D iterator v = iter([0, 1, 2, 0, 1, 2]) # 1D iterator v = iter([0, 1, 2, 3, iter([4, 5, 6, 7])]) # 2D iterator v = iter([iter([0, 1, 2, 3]), iter([4, 5, 6, 7])]) # 2D iterator v = iter([iter([0, 1, 2, 3]), # 3D iterator  iter([iter([4, 5]), iter([6, 7])])]) v = iter([iter([iter([0, 1]), iter([2, 3])]), # 3D iterator  iter([iter([4, 5]), iter([6, 7])])]) # No error  v = iter([1, 1.0, 1.0+0.0j, True]) v = iter(['A', b'A', bytearray(b'A'), 2, 2.3, 2.3+4.5j, True, [2, 3], (2, 3), {2, 3}, frozenset({2, 3}), {'A':'a'}, range(2, 3), iter([2, 3])]) for x in iter([0, 1, 2, 3, 4]): pass for x in iter([iter([0, 1, 2, 3]), iter([4, 5, 6, 7])]): pass for x in iter([iter([iter([0, 1]), iter([2, 3])]), iter([iter([4, 5]), iter([6, 7])])]): pass v1, v2, v3 = iter([0, 1, 2]) v1, *v2, v3 = iter([0, 1, 2, 3, 4, 5]) for v1, v2, v3 in iter([iter([0, 1, 2]), iter([3, 4, 5])]): pass for v1, *v2, v3 in iter([iter([0, 1, 2, 3, 4, 5]), iter([6, 7, 8, 9, 10, 11])]): pass print([*iter([0, 1, *iter([2])]), *iter([3, 4])]) print(*iter([0, 1, *iter([2])]), *iter([3, 4])) v = (x**2 for x in [0, 1, 2, 3, 4, 5, 6, 7]) v = ((y**2 for y in x) for x in [[0, 1, 2, 3], [4, 5, 6, 7]]) v = (((z**2 for z in y) for y in x) for x in [[[0, 1], [2, 3]], [[4, 5], [6, 7]]]) # No error  print(**iter([0, 1, 2, 3, 4])) v = iter(['A', 'B', 'C']) * 3 # Error
Enter fullscreen mode Exit fullscreen mode

<iter(object) & next(iterator)>: 

v = iter([0, 1, 2, 3, 4]) print(v) # <list_iterator object at 0x000002821F75D240>  print(type(v)) # <class 'list_iterator'>  print(next(v)) # 0 print(next(v)) # 1 print(next(v)) # 2 print(next(v)) # 3 print(next(v)) # 4 print(next(v)) # StopIteration:
Enter fullscreen mode Exit fullscreen mode

<__iter__() & __next__()>: 

v = [0, 1, 2, 3, 4].__iter__() print(v) # <list_iterator object at 0x000001FCD2883280>  print(v.__next__()) # 0 print(v.__next__()) # 1 print(v.__next__()) # 2 print(v.__next__()) # 3 print(v.__next__()) # 4 print(v.__next__()) # StopIteration:
Enter fullscreen mode Exit fullscreen mode

<iter(object) & next(iterator, default)>: 

v = iter([0, 1, 2, 3, 4]) print(v) # <list_iterator object at 0x000002821F75D240>  print(next(v, 'No value')) # 0 print(next(v, 'No value')) # 1 print(next(v, 'No value')) # 2 print(next(v, 'No value')) # 3 print(next(v, 'No value')) # 4 print(next(v, 'No value')) # No value
Enter fullscreen mode Exit fullscreen mode

<iter(object, sentinel) & for loop>: 

import random def get_random_numbers(): return random.randint(0, 9) v = iter(get_random_numbers, 3) print(v) # <callable_iterator object at 0x000001FCD2F376A0>  print(next(v)) # 7 print(next(v)) # 0 print(next(v)) # 2 print(next(v)) # 9 print(next(v)) # 5 print(next(v)) StopIteration:
Enter fullscreen mode Exit fullscreen mode

An iterator can be iterated with a for statement as shown below:

1D iterator: 

for x in iter([0, 1, 2, 3, 4]): print(x) # 0 # 1 # 2 # 3 # 4
Enter fullscreen mode Exit fullscreen mode

2D iterator: 

for x in iter([iter([0, 1, 2, 3]), iter([4, 5, 6, 7])]): for y in x: print(y) # 0 # 1 # 2 # 3 # 4 # 5 # 6 # 7
Enter fullscreen mode Exit fullscreen mode

3D iterator: 

for x in iter([iter([iter([0, 1]), iter([2, 3])]), iter([iter([4, 5]), iter([6, 7])])]): for y in x: for z in y: print(z) # 0 # 1 # 2 # 3 # 4 # 5 # 6 # 7
Enter fullscreen mode Exit fullscreen mode

An iterator can be unpacked with an assignment and for statement, function and * but not with ** as shown below:

v1, v2, v3 = iter([0, 1, 2]) print(v1, v2, v3) # 0 1 2
Enter fullscreen mode Exit fullscreen mode 
v1, *v2, v3 = iter([0, 1, 2, 3, 4, 5]) print(v1, v2, v3) # 0 [1, 2, 3, 4] 5 print(v1, *v2, v3) # 0 1 2 3 4 5
Enter fullscreen mode Exit fullscreen mode 
for v1, v2, v3 in iter([iter([0, 1, 2]), iter([3, 4, 5])]): print(v1, v2, v3) # 0 1 2 # 3 4 5
Enter fullscreen mode Exit fullscreen mode 
for v1, *v2, v3 in iter([iter([0, 1, 2, 3, 4, 5]), iter([6, 7, 8, 9, 10, 11])]): print(v1, v2, v3) print(v1, *v2, v3) # 0 [1, 2, 3, 4] 5 # 0 1 2 3 4 5 # 6 [7, 8, 9, 10] 11 # 6 7 8 9 10 11
Enter fullscreen mode Exit fullscreen mode 
def func(p1='a', p2='b', p3='c', p4='d', p5='e', p6='f'): print(p1, p2, p3, p4, p5, p6) func() # a b c d e f  func(*iter([0, 1, 2, 3]), *iter([4, 5])) # 0 1 2 3 4 5
Enter fullscreen mode Exit fullscreen mode 
def func(p1='a', p2='b', *args): print(p1, p2, args) print(p1, p2, *args) print(p1, p2, ['A', 'B', *args, 'C', 'D']) func() # a b () # a b # a b ['A', 'B', 'C', 'D']  func(*iter([0, 1, 2, 3]), *iter([4, 5])) # 0 1 (2, 3, 4, 5) # 0 1 2 3 4 5 # 0 1 ['A', 'B', 2, 3, 4, 5, 'C', 'D']
Enter fullscreen mode Exit fullscreen mode 
print([*iter([0, 1, *iter([2])]), *iter([3, 4])]) # [0, 1, 2, 3, 4]
Enter fullscreen mode Exit fullscreen mode 
print(*iter([0, 1, *iter([2]), *iter([3, 4])])) # 0 1 2 3 4
Enter fullscreen mode Exit fullscreen mode 
print(**iter([0, 1, 2, 3, 4])) # TypeError: print() argument after ** must be a mapping, not list_iterator
Enter fullscreen mode Exit fullscreen mode

An empty iterator isn't False as shown below:

print(bool(iter([]))) # Empty iterator # True  print(bool(iter([0]))) # iterator print(bool(iter([iter([])]))) # iterator(Empty iterator) # True
Enter fullscreen mode Exit fullscreen mode

A iterator can be checked if a specific element is or isn't in it with in keyword or not and in keyword respectively as shown below:

v = iter(['A', 'B', iter(['C', 'D'])]) print('B' in v) # True  print('C' in v) print(['C', 'D'] in v) print(iter(['C', 'D']) in v) # False
Enter fullscreen mode Exit fullscreen mode 
v = iter(['A', 'B', iter(['C', 'D'])]) print('B' not in v) # False  print('C' not in v) print(['C', 'D'] not in v) print(iter(['C', 'D']) not in v) # True
Enter fullscreen mode Exit fullscreen mode

A iterator cannot be enlarged with * and a number as shown below:

v = iter(['A', 'B', 'C']) * 3 print(v) # TypeError: unsupported operand type(s) for *: 'list_iterator' and 'int'
Enter fullscreen mode Exit fullscreen mode

An iterator can be used with len() after using list(), tuple(), set(), frozenset() or more_itertools.ilen() to get the length as shown below:

*Memo:

  • more-itertools must be installed with pip install more-itertools.
  • An iterator cannot be directly used with len() to get the length. 
from copy import copy from more_itertools import ilen v1 = iter([0, 1, 2, 3, 4]) v2 = copy(v1) print(len(list(v2))) # 5  v2 = copy(v1) print(len(tuple(v2))) # 5  v2 = copy(v1) print(len(set(v2))) # 5  v2 = copy(v1) print(len(frozenset(v2))) # 5  v2 = copy(v1) print(ilen(v2)) # 5  v2 = copy(v1) print(len(v2)) # TypeError: object of type 'list_iterator' has no len()
Enter fullscreen mode Exit fullscreen mode

An iterator cannot be read or changed by indexing or slicing as shown below:

*Memo:

  • A del statement can still be used to remove one or more variables themselves. 
v = iter(['a', 'b', 'c', 'd', 'e', 'f']) print(v[0], v[2:6]) # TypeError: 'list_iterator' object is not subscriptable
Enter fullscreen mode Exit fullscreen mode 
v = iter(['a', 'b', 'c', 'd', 'e', 'f']) v[0] = 'X' v[2:6] = ['Y', 'Z'] # TypeError: 'list_iterator' object does not support item assignment
Enter fullscreen mode Exit fullscreen mode 
v = iter(['a', 'b', 'c', 'd', 'e', 'f']) del v[0], v[3:5] # TypeError: 'list_iterator' object does not support item deletion
Enter fullscreen mode Exit fullscreen mode 
v = iter(['a', 'b', 'c', 'd', 'e', 'f']) del v print(v) # NameError: name 'v' is not defined
Enter fullscreen mode Exit fullscreen mode

If you really want to read and change an iterator, use list() and iter() or __iter__() as shown below:

v = iter(['a', 'b', 'c', 'd', 'e', 'f']) v = list(v) v[0] = 'X' v[2:6] = ['Y', 'Z'] v = iter(v) v = v.__iter__() for x in v: print(x) # X # b # Y # Z
Enter fullscreen mode Exit fullscreen mode 
v = iter(['a', 'b', 'c', 'd', 'e', 'f']) v = list(v) del v[0], v[3:5] v = iter(v) v = v.__iter__() for x in v: print(x) # b # c # d
Enter fullscreen mode Exit fullscreen mode

Top comments (0)