Summary of Usage of Common Functions of python Pool

  • 2021-10-27 08:26:02
  • OfStack

1. Description

apply_async (func [, args [, kwds]): Call func using non-blocking (parallel execution, blocking mode must wait for the last procedure to exit before executing the next procedure), args is the parameter list transmitted to func, and kwds is the keyword parameter list transmitted to func.

close (): Shut down Pool so that it does not accept new tasks.

terminate (): Terminate the task immediately whether it is completed or not.

join (): The main process is blocked, waiting for the child process to exit, and must be used after close or terminate.

2. Examples


#coding: utf-8
import multiprocessing
import time
 
 
def func(msg):
    print("msg:", msg)
    time.sleep(3)
    print("end")
 
 
if __name__ == "__main__":
    cores = multiprocessing.cpu_count()
    pool = multiprocessing.Pool(processes=cores)
    print("Adding tasks...")
    for i in range(cores):
        msg = "hello %d" %(i)
        pool.apply_async(func, (msg, ))   # The total number of processes maintained is processes , when 1 A new process will be added after the execution of each process 
    print("Starting tasks...")
    pool.close()
    pool.join()   # Call join Before calling the close Function, otherwise an error will occur. End of execution close No new processes will be added after pool,join Function waits for all child processes to end 
    print("Sub-process(es) done.")

Instance extension:


# -*- coding:utf-8 -*-
from multiprocessing import Pool
import os, time, random
def worker(msg):
  t_start = time.time()
  print("%s Start execution , The process number is %d" % (msg,os.getpid()))
  # random.random() Random generation 0~1 Floating point number between 
  time.sleep(random.random()*2) 
  t_stop = time.time()
  print(msg," Complete execution, time consuming %0.2f" % (t_stop-t_start))
po = Pool(3) #  Definition 1 Pool of processes, maximum number of processes 3
for i in range(0,10):
  # Pool().apply_async( Target to call ,( Parameter ancestor passed to the target ,))
  #  Each loop will use the free child process to call the target 
  po.apply_async(worker,(i,))
print("----start----")
po.close() #  Close the process pool. After closing, po No new requests will be received 
po.join() #  Wait po The execution of all child processes in is completed, and must be placed in the close Statement 
print("-----end-----")

Running result

----start----
0, process number 21466
1, process number 21468
2, process number 21467
0 completed, taking 1.01
3, process number 21466
2 completed, taking 1.24%
4, process number 21467
3 completed, taking 0.56%
5, process number 21466
1 completed, taking 1.68% time
6, process number 21468
4 completed, taking 0.67% time
7, process number 21467
5 completed, taking 0.83% time
8, process number 21466
6 completed, taking 0.75% time
9, process number 21468
7 completed, taking 1.03
8 completed, taking 1.05%
9 completed, taking 1.69%
-----end-----


Related articles: