A simple understanding of what an Python generator is

  • 2021-07-09 08:27:44
  • OfStack

Preface

Generator is one of the most difficult concepts for Python beginner developers to understand. Although it is considered as an advanced skill in Python programming, it can be seen everywhere in various projects, so you have to understand it, use it or even fall in love with it.

When it comes to generators, it is inevitable to pull out iterators. In contrast, a generator is an object that is very similar in behavior to an iterator. If an iterator is compared to an Android system, then a generator is iOS. The two are similar in function, but the generator is more elegant.

What is an iterator
As the name implies, an iterator is an object used for iterative operations (for loops) that iteratively fetches every one of its elements, as shown in Listing 1, and any object that implements the __next__ method (python2 is next) can be called an iterator.

It differs from a list in that instead of loading all elements into memory once like a list, the iterator is built to return elements in a delayed calculation (lazy evaluation), which is its advantage. For example, the list contains 10 million integers and needs more than 400M of memory, while the iterator only needs a few 10 bytes of space. Because it doesn't load all the elements into memory, it waits until the next method is called before returning the element (the way call by need is called on demand, essentially the for loop is the next method of the iterator constantly called).

Take Fibonacci array as an example to implement an iterator:


class Fib:
def __init__(self, n):
self.prev = 0
self.cur = 1
self.n = n
def __iter__(self):
return self
def __next__(self):
if self.n > 0:
value = self.cur
self.cur = self.cur + self.prev
self.prev = value
self.n -= 1
return value
else:
raise StopIteration()
#  Compatible python2
def __next__(self):
return self.next()
f = Fib(10)
print([i for i in f])
#[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

What is a generator

Once you know the iterator, you can officially enter the topic of generator. Ordinary functions return 1 value with return, Is the same as other languages such as Java, However, in Python there is also a function, using the keyword yield to return value, this function is called generator function, when the function is called, it will return a generator object, the generator is essentially an iterator, is also used in iterative operations, so it has and iterator 1-like characteristics, only 1 difference is not 1-like implementation, the latter is more concise

The simplest generator function:


>>> def func(n):
... yield n*2
...
>>> func
<function func at 0x00000000029F6EB8>
>>> g = func(5)
>>> g
<generator object func at 0x0000000002908630>
>>>

func is a generator function, and the returned object when calling this function is generator g. The behavior of this generator object is very similar to that of iterator, and can be used in scenarios such as for loop. Note that the value corresponding to yield does not return immediately when the function is called, but only when the next method is called (essentially the for loop also calls the next method)


>>> g = func(5)
>>> next(g)
10
>>> g = func(5)
>>> for i in g:
... print(i)
...
10

Then why use a generator? Obviously, using a generator is several levels higher than an iterator in compaction, it doesn't have so much tedious code, and its performance is as efficient as 1, so why not use it? Let's see how easy it is to implement Fibonacci sequences with generators.


def fib(n):
prev, curr = 0, 1
while n > 0:
n -= 1
yield curr
prev, curr = curr, curr + prev
print([i for i in fib(10)])
#[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

Generator expression

The list derivation (list comprehension) was introduced in the previous article "Writing code in this way is more elegant". The generator expression is very similar to the list derivation, but they return different objects. The former returns the generator object and the latter returns the list object.


>>> g = (x*2 for x in range(10))
>>> type(g)
<type 'generator'>
>>> l = [x*2 for x in range(10)]
>>> type(l)
<type 'list'>

The advantage of generators, which have been described earlier, is that they are obviously more suitable when iterating over massive amounts of data.


Related articles: