Python iteration and iterator elaboration

The iterator

An iterator (iterator), sometimes called a cursor (cursor), is a software design pattern for programming that allows the designer to traverse an interface over a container object (container, such as a linked list or array) without worrying about the implementation details of the container object's memory allocation.

From wikipedia

In other words, an iterator is like a cursor, which can be used to access the elements of an iterable object. Also, Python isn't the only one with this feature. For example, C++ also has this in STL, vector < int > : : iterator it. Let's focus on the iterable objects and iterators in Python.

Python iterable objects (Iterable)

In Python, for is often used to traverse an object. At this time, the object being traversed is an iterable object, such as list and tuple. If give a precise definition, is defined as long as it can return an iterator __iter__ method, or defines the __getitem__ can support the subscript index method (the double underline method can fully explain in the other chapters), then it is an object iteration.

Python iterator (iterator)

The iterator is implemented by next(), which returns the next element every time it is called, and an StopIteration exception when there is no next element, so it is actually an iterator that defines this method. You can use the following example to experience the following iterator:

In [38]: s = 'ab'

In [39]: it = iter(s)

In [40]: it
Out[40]: <iterator at 0x1068e6d50>

In [41]: print it
<iterator object at 0x1068e6d50>

In [42]: it.next()
Out[42]: 'a'

In [43]: it.next()
Out[43]: 'b'

In [44]: it.next()
---------------------------------------------------------------------------
StopIteration               Traceback (most recent call last)
<ipython-input-44-54f0920595b2> in <module>()
----> 1 it.next()

StopIteration:

I implemented an iterator by myself, as follows (see the official website documentation) :

class Reverse:
  """Iterator for looping over a sequence backwards."""
  def __init__(self, data):
    self.data = data
    self.index = len(data)

  def __iter__(self):
    return self

  def next(self):
    if self.index == 0:
      raise StopIteration
    self.index = self.index - 1
    return self.data[self.index]

rev = Reverse('spam')
for char in rev:
  print char

[output]
m
a
p
s

Generator (Generators)

Generators are the simplest and most powerful tool for building iterators. Unlike normal functions, yield is used instead of return when a value is returned, and yield automatically builds next() and iter(). Isn't it easy? Such as:

def reverse(data):
  for index in range(len(data)-1, -1, -1):
    yield data[index]

>>> for char in reverse('golf'):
...   print char
...
f
l
o
g

The best scenario for generators is that you don't want to allocate all of the computed large result sets to memory at the same time, especially if the result sets also contain loops. For example, if we were to loop through 1,000,000 Numbers, we would normally use xrange() instead of range(), since the former returns a generator and the latter a list (which consumes a lot of space).

Help on built-in function range in module __builtin__:

range(...)
  range(stop) -> list of integers
  range(start, stop[, step]) -> list of integers

  Return a list containing an arithmetic progression of integers.
  range(i, j) returns [i, i+1, i+2, ..., j-1]; start (!) defaults to 0.
  When step is given, it specifies the increment (or decrement).
  For example, range(4) returns [0, 1, 2, 3]. The end point is omitted!
  These are exactly the valid indices for a list of 4 elements.

class xrange(object)
 | xrange(stop) -> xrange object
 | xrange(start, stop[, step]) -> xrange object
 |
 | Like range(), but instead of returning a list, returns an object that
 | generates the numbers in the range on demand. For looping, this is
 | slightly faster than range() and more memory efficient.
iter()

Converts an iterable object into an iterator.

In [113]: s = 'abc'

In [114]: s.next()
---------------------------------------------------------------------------
AttributeError              Traceback (most recent call last)
<ipython-input-114-5e5e6532ea26> in <module>()
----> 1 s.next()

AttributeError: 'str' object has no attribute 'next'

In [115]: it = iter(s)

In [116]: it.next()
Out[116]: 'a'

Generator expression

The only difference from list derivation is that the brackets are replaced with parentheses, as follows:

In [119]: num = (i for i in range(10))

In [120]: sum(num)
Out[120]: 45