The usual Python decorators iterators and generators

While learning python, the three "nominators" can be a bit of a challenge for people with no programming experience in other languages. This blog explains the blogger's own understanding of decorators, iterators, and generators.

Why use decorators

What is a decorator? "Decoration" from the literal meaning to who is on a particular building according to the fixed way of thinking and style of beautification of 1 kind of behavior, the so-called "device" is a tool, for python decorator is to be able to without modifying the original code to add new functions, such as 1 software online, we need not to modify the source code and the way of change is called the circumstance also can add a new function, in python decorators can be used to realize, when writing the code also want to consider the scalability, below we look to see steps 1 1 1 python decorator.

A simple example introduces a parameter-free decorator

Look at a few simple lines of code. The code runs to sleep for 2 seconds before printing "hello boy!" :

import time
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo()

We now need to add a program timing function to it, but cannot modify the original code:

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

Look! We achieved this without modifying the original code, and because the function is also an object, we were able to pass the function foo as an argument to the function timmer.

In python, there is a simpler alternative to foo=timmer(foo), using @timmer, which in python is called grammar sugar.

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
@timmer  # Is equal to the  foo=timmer(foo)
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo()

Let's follow the execution process of the analysis function step by step:

1. Import time module

import time

2. Define timmer, which does not execute the code inside the function

def timmer(func):

3. Call the decorator, equivalent to foo=timer(foo), is the function foo as a parameter to the function timmer

@timmer

4. Run the function timmer, taking the parameter func=foo

def timmer(func):

5. In the function timmer, the function wrapper is defined, and the internal code of the function wrapper is not executed, and then the function wrapper is returned as the return value

return wrapper

6. Assigns the return value to foo, in step 3, foo=timmer(foo), remember

@timmer # Is equal to the  foo=timmer(foo)

7. Run function foo (), but the function is not the original function, can print foo, right, because before we gave the foo wrapper as return values, so perform foo here is in the execution wrapper, again in order to determine this 1 point, you can also print wrapper their memory address is the same, so are all pointing to one address space:

<function timmer.<locals>.wrapper at 0x00000180E0A8A950> # print foo The results of the 
<function timmer.<locals>.wrapper at 0x000001F10AD8A950> # print wrapper The results of the 
foo()

8. Run the function wrapper, record the start time, and execute the function func. In step 4, func is assigned by foo.

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

0

9. Record the end time, print the running time and finish the program.

Hello boy!
Run time is 2.000161 
 

There are reference decorators

In the previous example, the original function has no arguments. Now let's look at 1. When the original function has arguments, how to modify the decorator function?

import time
def timmer(func):
 def wrapper(*args,**kwargs):
  """ Timing function """
  start_time=time.time()
  res=func(*args,**kwargs)
  end_time=time.time()
  print("Run time is %f"%(end_time-start_time))
  return res
 return wrapper
@timmer 
def my_max(x,y):
 """ Returns the maximum of two values """
 res=x if x > y else y
 time.sleep(2)
 return res
res=my_max(1,2)
print(res)
# The results 
Run time is 2.000175

In this example, my_max has two positions that need to be passed in when the original function needs to pass in an argument. You only need to add two formal parameters to wrapper. Variable arguments (*args,**kwargs) can also be used in this example.

Let's take a look at a decorator with parameters:

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

3

If the decorator itself has parameters, an additional layer of embedded functions is needed. Let's analyze the execution process step by step:

Define the function auth

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

4

2. To invoke the interpreter, first run the function auth(filetype='file')

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

5

3. Run the function auth, which defines a function auth2 and returns it as a return value, then @auth (filetype='file') is the same as @auth2, which is the same as index=auth2(index)

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

6

auth2(index) executes, func=index, defines the function wrapper, and returns it. At this time, index is equal to wrapper

def wrapper(*args,**kwargs):
return wrapper

5. When index is run, that is, wrapper, run the internal code of the function, filetype=="file", prompt the user to output the user name and password to determine whether the input is correct, if correct, then execute the function func(), equal to execute the original index and print

if filetype == "file":
    username=input("Please input your username:")
    passwd=input("Please input your password : ")
    if passwd == '123456' and username == 'Frank':
     print("Login successful")
     func()

6. Run the results test

import time
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
 return wrapper
def foo():
 """ print """
 time.sleep(2)
 print("Hello boy!")
foo=timmer(foo)
foo()
# The results 
Hello boy!
Run time is 2.000446 

9

Decorators can also be superimposed:

import time
#
def timmer(func):
 def wrapper():
  """ Timing function """
  time_start=time.time()
  func()
  time_end=time.time()
  print("Run time is %f "%(time_end-time_start))
  # print("---",wrapper)
 return wrapper
def auth(filetype):
 def auth2(func):
  def wrapper(*args,**kwargs):
   if filetype == "file":
    username=input("Please input your username:")
    passwd=input("Please input your password : ")
    if passwd == '123456' and username == 'Frank':
     print("Login successful")
     func()
    else:
     print("login error!")
   if filetype == 'SQL':
    print("No SQL")
  return wrapper
 return auth2
@timmer
@auth(filetype='file') # First to return 1 a auth2 == " @auth2 == "  index=auth2() == "  index=wrapper
def index():
 print("Welcome to China")
index()

# The test results 
Please input your username:Frank
Please input your password : 123456
Login successful
Welcome to China
Run time is 7.966267

Annotation to optimize

import time
def timmer(func):
 def wrapper():
  """ Calculate program running time """
  start_time=time.time()
  func()
  end_time=time.time()
  print("Run time is %s:"%(end_time-start_time))
 return wrapper
@timmer
def my_index():
 """ Print welcome """
 time.sleep(1)
 print("Welcome to China!")
my_index()
print(my_index.__doc__)

# The results 
Welcome to China!
Run time is 1.0005640983581543:
 Calculate program running time 

When we use the decorator, we don't change the code itself, but when we run it, like in the example above, running my_index is actually running wrapper, if we print comments for my_index, we print comments for wrapper(), how do we optimize it?

wraps can be imported into module functools, as shown below:

import time
from functools import wraps
def timmer(func):
 @wraps(func)
 def wrapper():
  """ Calculate program running time """
  start_time=time.time()
  func()
  end_time=time.time()
  print("Run time is %s:"%(end_time-start_time))
 return wrapper
@timmer
def my_index():
 """ Print welcome """
 time.sleep(1)
 print("Welcome to China!")
my_index()
print(my_index.__doc__)
# The results 
Welcome to China!
Run time is 1.0003223419189453:
 Print welcome 

Thus, on the surface, nothing has changed in the original function.

Why use iterators

In python, iterator can be used to describe the advantages and disadvantages of iterator 1. If you don't understand it, you can skip it first. After reading this blog, I believe you will understand the meaning:

Advantages:

Iterators do not depend on indexes when they are evaluated, so they can iterate over objects that do not have indexes, such as dictionaries and files

Iterators are lazy computations and more memory efficient than lists

Disadvantages:

Unable to get iterator length, no list flexibility

You can only evaluate it backwards, not backwards

What is an iterator

So what is an iterator in python?

If the object has ___ 205en__ (), it is iteratable, with the iterator using the function next()

Let's look at a simple iterator:

my_list=[1,2,3]
li=iter(my_list)  #li=my_list.__iter__()
print(li)
print(next(li))
print(next(li))
print(next(li))
# The results 
<list_iterator object at 0x000002591652C470>
2

As can be seen, the built-in function iter can be used to convert the list into a list iterator. next() is used to obtain the value. Once the value is obtained, a value is taken.

my_list=[1,2,3]
li=iter(my_list)
while True:
 try:
  print(next(li))
 except StopIteration:
  print("Over")
  break
 else:
  print("get!")
# The results 
get!
get!
get!
Over

View iterable objects and iterator objects

The Iterable module can be used to determine whether an object is iterable:

from collections import Iterable
s="hello" # Define string 
l=[1,2,3,4] # Define a list 
t=(1,2,3) # Define a tuple 
d={'a':1} # Define a dictionary 
set1={1,2,3,4} # Define a collection 
f=open("a.txt") # Define text 
#  See if they are all iteratable 
print(isinstance(s,Iterable))
print(isinstance(l,Iterable))
print(isinstance(t,Iterable))
print(isinstance(d,Iterable))
print(isinstance(set1,Iterable))
print(isinstance(f,Iterable))
# The results 
True
True
True
True
True
True

By judging, we can be sure that the commonly used data types we know can be iterated.

The Iterator module can be used to determine whether an object is an iterator:

from collections import Iterable,Iterator
s="hello"
l=[1,2,3,4]
t=(1,2,3)
d={'a':1}
set1={1,2,3,4}
f=open("a.txt")
#  See if they are all iteratable 
print(isinstance(s,Iterator))
print(isinstance(l,Iterator))
print(isinstance(t,Iterator))
print(isinstance(d,Iterator))
print(isinstance(set1,Iterator))
print(isinstance(f,Iterator))
# The results 
False
False
False
False
False
True

You know that only the files are iterators, so you can use next() directly without needing to convert to iterators.

What is a generator

The producer is a function with yield

Let's look at a simple generator

def my_yield():
 print('first')
 yield 1
g=my_yield()
print(g)
# The results 
<generator object my_yield at 0x0000024366D7E258>

The generator is also an iterator

from collections import Iterator
def my_yield():
 print('first')
 yield 1
g=my_yield()
print(isinstance(g,Iterator))
# The results 
True

So you could use next()

print(next(g))
# The results 
first
1
 

The execution of a generator

Let's take a look at the following example to understand the execution process of production

def my_yield():
 print('first')
 yield 1
 print('second')
 yield 2
 print('Third')
 yield 3
g=my_yield()
next(g)
next(g)
next(g)
# The results 
first
second
Third

1. Define the generator my_yield and assign it to g

def my_yield():
g=my_yield()

2. Start the first execution of next(), start the producer function, print statement 1, pause when yileld is encountered, and return a 1, which will be shown here if you want to print the return value

 print('first')
 yield 1

3. Execute 2 more times and print the string (pause 1 time for each execution)

 print('second')
 yield 2
 print('Third')
 yield 3

4. If you add next() once more, the StopIteration anomaly will be reported

Each time the generator pauses, the state of the function is saved. Here's an example:

def foo():
 i=0
 while True:
  yield i
  i+=1
g=foo()
for num in g:
 if num < 10:
  print(num)
 else:
  break
# The results 

next() is implied in the for loop. Every next1 is paused once, and the if statement determines once, and then executes the next next. It can be seen that our while loop does not go on indefinitely, but the state is preserved.

Coroutines function

Let's look at the generator and execution results below

def eater(name):
 print('%s start to eat food'%name)
 while True:
  food=yield
  print('%s get %s ,to start eat'%(name,food))
 print('done')
e=eater('Frank')
next(e)
e.send('egg') # to yield send 1 Value and continue to execute the code 
e.send('tomato')
# The results 
Frank start to eat food
Frank get egg ,to start eat
Frank get tomato ,to start eat

send can be directly transmitted to yield. The function containing yield expression is also called the coroutine function.

When running the program, do not use send directly, you must first use next() initializer.

If there are more than one such function, we will go to next()1 each time. In case we forget this step, we can initialize it with the decorator:

def init(func):
 def wrapper(*args):
  res = func(*args)
  next(res)  #  So let's do it here next
  return res
 return wrapper
@init
def eater(name):
 print('%s start to eat food'%name)
 while True:
  food=yield
  print('%s get %s ,to start eat'%(name,food))
 print('done')
e=eater('Frank')
e.send('egg') 
e.send('tomato')

So if there are more generators in the program that need to be initialized, simply call the decorator.