Python iterator and generator examples

This paper illustrates the iterators and generators of python with examples, as follows:

1. Overview of iterators:
 
Iterators are a way to access collection elements. The iterator object is accessed from the first element of the collection until all elements are accessed. Iterators can only move forward and not backward, which is fine, because people rarely move backward in an iteration.
 
1.1 advantages of using iterators
 
For data structures (such as tuple, list) that have native support for random access, iterators have no advantage over the index access of the classic for loop and instead lose the index value (which can be retrieved using the built-in function enumerate()). But for data structures (such as sets) that cannot be accessed at random, iterators are the only way to access elements.

In addition, one of the great advantages of iterators is that they do not require all the elements of the entire iteration to be prepared in advance. Iterators evaluate an element only when it is iterated over, and before or after that, the element may not exist or be destroyed. This feature makes it particularly suitable for traversing large or infinite collections, such as files of several gigabytes, or Fibonacci sequences, etc.

Iterators have the greater benefit of providing a uniform interface for accessing collections, which can be accessed using iterators as long as the method objects are defined with iteration.
 
Iterators have two basic methods
 
Next method: returns the next element of the iterator
Method: returns the iterator object itself
Following is an example of generating Fibonacci Numbers to show why iterators are used
 
Sample code 1

 def fab(max): 
  n, a, b = 0, 0, 1 
  while n < max: 
    print b 
    a, b = b, a + b 
    n = n + 1

Printing print directly in the function fab(Max) results in poor reusability of the function because fab returns None. Other functions cannot get the array returned by the fab function.
 
Sample code 2

 def fab(max): 
  L = []
  n, a, b = 0, 0, 1 
  while n < max: 
    L.append(b) 
    a, b = b, a + b 
    n = n + 1
  return L

Code 2 satisfies the reusability requirement, but it takes up memory space, preferably not.
 
Sample code 3
 
Contrast:
 

for i in range(1000): pass
for i in xrange(1000): pass

The first returns a list of 1000 elements, and the second returns one element per iteration, so iterators can be used to solve the problem of reusing occupied space
 

 class Fab(object): 
  def __init__(self, max): 
    self.max = max 
    self.n, self.a, self.b = 0, 0, 1 
 
  def __iter__(self): 
    return self 
 
  def next(self): 
    if self.n < self.max: 
      r = self.b 
      self.a, self.b = self.b, self.a + self.b 
      self.n = self.n + 1 
      return r 
    raise StopIteration()

perform

>>> for key in Fabs(5):
  print key

The Fabs class keeps returning the next number in the sequence through next(), always with a constant footprint

1.2 use iterators

Iter (iterable) can be used to obtain iterator objects:

>>> lst = range(5)
>>> it = iter(lst)
>>> it
<listiterator object at 0x01A63110>

Use the next() method to access the next element:

>>> it.next()
 
>>> it.next()
 
>>> it.next()

Python handles iterator overbounds by throwing a StopIteration exception

>>> it.next()
 
>>> it.next
<method-wrapper 'next' of listiterator object at 0x01A63110>
>>> it.next()
 
>>> it.next()
 
Traceback (most recent call last):
 File "<pyshell#27>", line 1, in <module>
  it.next()
StopIteration

Now that you know StopIteration, you can use the iterator to iterate over it

lst = range(5)
it = iter(lst)
try:
  while True:
    val = it.next()
    print val
except StopIteration:
  pass

In fact, because iterators are so common, python makes iterator syntax sugar specifically for the for keyword. In the for loop, Python automatically calls the factory function iter() to get the iterator, automatically calls next() to get the element, and checks the StopIteration exception. The following

>>> a = (1, 2, 3, 4)
>>> for key in a:
  print key

Python first calls the iter function iterator on the object after the keyword in, then calls the iterator's next method to get the element, until it throws a StopIteration exception.

1.3 define iterators
 
Here's an example -- the Fibonacci sequence
 

# -*- coding: cp936 -*-
class Fabs(object):
  def __init__(self,max):
    self.max = max
    self.n, self.a, self.b = 0, 0, 1 # In particular: no 0 Item is 0 In the first 1 The term is the first one 1. The whole sequence from 1 start 
  def __iter__(self):
    return self
  def next(self):
    if self.n < self.max:
      r = self.b
      self.a, self.b = self.b, self.a + self.b
      self.n = self.n + 1
      return r
    raise StopIteration()
 
print Fabs(5)
for key in Fabs(5):
  print key

The results of

<__main__.Fabs object at 0x01A63090>

2. The iterator

A function with yield is called a generator in Python, with a few examples (again, Fibonacci)
 
As you can see, code 3 is not nearly as clean as code 1, and the generator (yield) keeps code 1 simple and code 3 clean
 
Sample code 4
 

def fab(max):
  n, a, b = 0, 0, 1
  while n < max:
    yield b
    a, b = b, a + b
    n = n = 1

perform

>>> for n in fab(5):
  print n

Simply put, the effect of yield is to turn a function into an generator. A function with yield is no longer a normal function. The Python interpreter treats it as an generator. In the for loop is executed, each loop will execute code within the overall function, execution to yield b, the overall function returns an iterator values, the next iteration, the code from the yield of the next statement b continue to perform, and the function of the local variable seems to interrupt execution and the last is exactly the same as before, so the function to continue, until once again hit yield. It looks as if a function is interrupted by yield several times during normal execution, with each interrupt returning the current iteration value via yield.
 
You can also manually call the next() method of fab(5) (because fab(5) is a generator object with the next() method), so that we can see the execution flow of fab more clearly:

>>> f = fab(3)
>>> f.next()
1
>>> f.next()
1
>>> f.next()
2
>>> f.next()
 
Traceback (most recent call last):
 File "<pyshell#62>", line 1, in <module>
  f.next()
StopIteration

The return effect

In a generator, if there is no return, the function is executed by default. If a return is encountered, if it is returned during execution, a StopIteration is simply thrown to terminate the iteration. For example,
 

>>> s = fab(5)
>>> s.next()
1
>>> s.next()
 
Traceback (most recent call last):
 File "<pyshell#66>", line 1, in <module>
  s.next()
StopIteration

Example code 5   File to read

 def read_file(fpath): 
  BLOCK_SIZE = 1024 
  with open(fpath, 'rb') as f: 
    while True: 
      block = f.read(BLOCK_SIZE) 
      if block: 
        yield block 
      else: 
        return

If you call the read() method directly on the file object, it results in unpredictable memory usage. A good approach is to use a fixed-length buffer to read the contents of the file over and over again. With yield, we no longer need to write an iterative class to read the file, so we can easily read the file.