Python generator (Generator) details

With the list generator, we can create a list directly. However, due to memory constraints, the list capacity is certainly limited. In addition, creating a list of 1 million elements takes up a lot of storage space. If we only need to access the first few elements, most of the space taken by the next elements will be wasted.

So, if the list elements can be inferred by some algorithm, can we continue to infer the next elements as we go through the loop? This saves a lot of space by eliminating the need to create a full list. In Python, this one-side loop one-side calculation mechanism is called a generator (Generator).

Simple generator

There are several ways to create an generator. The first method is as simple as changing [] to () for a list generator to create an generator:

>>> L = [x * x for x in range(10)]

>>> L

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

>>> g = (x * x for x in range(10))

>>> g

<generator object <genexpr> at 0x104feab40>

The difference between creating L and g is only in the outermost [] and (), where L is one list and g is one generator.
We can print out every element of list, but how do we print out every element of generator?

If you want to print out one by one, you can do so using generator's next() method:

>>> g.next()

0

>>> g.next()

1

>>> g.next()

4

>>> g.next()

9

>>> g.next()

16

>>> g.next()

25

>>> g.next()

36

>>> g.next()

49

>>> g.next()

64

>>> g.next()

81

>>> g.next()

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

StopIteration

As we said, generator holds the algorithm, and every time you call next(), you calculate the value of the next element, until you get to the last element, and when there are no more elements, you throw an StopIteration error.

Of course, the above method of constantly calling next() is really abnormal. The correct method is to use for loop, because generator is also an iterable object:

>>> g = (x * x for x in range(10))

>>> for n in g:

...     print n

...

0

1

4

9

16

25

36

49

64

81

So, after we create an generator, we basically never call the next() method, but iterate through the for loop.

Generator with yield statement

After careful observation, it can be seen that the fib function actually defines the calculation rules of the fiboracci sequence, which can start from the first element and then calculate any subsequent element. This logic is actually very similar to generator.

In other words, the function above is only one step away from generator. To change the fib function to generator, simply change print b to yield b:

def fib(max):

    n, a, b = 0, 0, 1

    while n < max:

        yield b

        a, b = b, a + b

        n = n + 1

This is another way to define generator. If a function definition contains the yield keyword, then the function is no longer an ordinary function, but an generator:

>>> fib(6)

<generator object fib at 0x104feaaa0>

Here, the hardest thing to understand is that generator and the function execution flow are different. The function is executed sequentially, returning either the return statement or the last line of the function statement. The function that becomes generator is executed each time next() is called, and the yield statement is returned when it is encountered, and the yield statement returned last time is resumed when it is executed again.

For a simple example, define 1 generator and return the Numbers 1, 3, 5 in turn:

>>> def odd():

...     print 'step 1'

...     yield 1

...     print 'step 2'

...     yield 3

...     print 'step 3'

...     yield 5

...

>>> o = odd()

>>> o.next()

step 1

1

>>> o.next()

step 2

3

>>> o.next()

step 3

5

>>> o.next()

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

StopIteration

As you can see, odd is not a normal function, but generator. In the process of execution, when it encounters yield, it will be interrupted, and the execution will continue the next time. After three yield executions, there is no yield left to execute, so the fourth call to next() reports an error.

Going back to the example of fib, we call yield over and over again in the loop, and it keeps breaking. Of course you have to put a condition on the loop to get out of the loop, otherwise you'll have an infinite list of Numbers.

Similarly, after changing the function to generator, we basically never call it with next(), but simply use the for loop to iterate:

>>> for n in fib(6):

...     print n

...

1

1

2

3

5

8

Enhanced generator

In python2.5,1 of the enhancements are added to the generator, so in addition to next() to get the next generated value, the user can send the value back to the generator [send()], throw an exception in the generator, and ask the generator to exit [close()]

def gen(x):

    count = x

    while True:

        val = (yield count) 

        if val is not None:

            count = val

        else:

            count += 1
f = gen(5)

print f.next()

print f.next()

print f.next()

print '===================='

print f.send(9)# Send digital 9 For the generator 

print f.next()

print f.next()

The output

5

6

7

====================

9

10

11