Multithreading is used in the python callback function

demo below is a simple test script written as required

#!/usr/bin/env python
# coding: utf-8
#  The first 1 A list of dependent components and version Numbers followed by the owner name 
#  And then there is regulation 2 Since the establishment of the list, the person in charge is empty 
#  Therefore, according to the requirements of the component, version number, responsible for different processing 
#  At this moment in for In cycle basis if Judge, write callback function processing 
#  Format is not 1 The test data to the data 
a = [[u'tool-1', u'1.9.13'], u'xiaowang', u'xiaoqu', [u'tool-2', u'1.9.23'], [u'tool-3', u'1.9.33'], [u'tool-4', u'1.9.43'], u'pi',[u'tool-5', u'1.9.53']]
# a = [[u'tool-1', u'1.9.13'],u'xiaowang',[u'tool-2', u'1.9.23'],u'xiaowang', [u'tool-3', u'1.9.33'],u'xiaowang']
# a = [[u'tool-1', u'1.9.13']]
# [u'tool-1', u'1.9.13']
your_pro = a[0]
# print your_pro
# [u'xiaowang', u'xiaoqu', [u'tool-2', u'1.9.23']]
tmp = a[1:]
# print tmp
def git_callback(whole_v, proj_value, name_value):
  #  If there is a responsible person there is 
  try:
    if type(name_value[0]) is unicode:
      #  Pair remove list 0 Data following the data of the index (depending on the name and version number) is traversed 
      for i in name_value:
        #  The following data is a list of the callback 
        if type(i) is list:
          tmp_index = whole_v.index(i)+1
          return git_callback(whole_v, whole_v[whole_v.index(i)], whole_v[tmp_index:])
        else:
          #  Print dependencies, version Numbers   head   start 
          print proj_value+i.split()+['start']
    else:
      #  If the component followed by the owner has empty list data in this format 
      #  That is, only the dependency and version number list data, the owner is empty, the dependent version number is printed 
      ver = proj_value
      owner = name_value
      if type(owner[0]) is unicode:
        return git_callback(whole_v, ver, owner)
      else:
        print ver
        #  I'm just trying to figure out if I'm going to the end of the list 1 position 
        #  If it's at the end 1 A value that is not a string Unicode It's a list 
        #  Just print out the item 
        if whole_v.index(owner[0]) == len(whole_v)-1:
          #  Print the final 1 A value 
          print whole_v[-1:]
        else:
          #  It's a little bit convoluted here, so let's print and debug ...
          new_ver = whole_v[whole_v.index(ver)+1]
          owner = whole_v[whole_v.index(ver)+2:]
          return git_callback(whole_v, new_ver, owner)
  except IndexError as e:
    print proj_value
    print e
git_callback(a, your_pro, tmp)

demo output:

Boom:git_response pirogue$ python test.py
[u'tool-1', u'1.9.13', u'xiaowang', 'start']
[u'tool-1', u'1.9.13', u'xiaoqu', 'start']
[u'tool-2', u'1.9.23']
[u'tool-3', u'1.9.33']
[u'tool-4', u'1.9.43', u'pi', 'start']
[u'tool-5', u'1.9.53']
list index out of range

Multithreading for python

The following code is a snippet from the main program

from multiprocessing.dummy import Pool as ThreadPool
#  judge git The query returns more than one dependent data format 1 The callback 
def git_callback(whole_v, proj_value, name_value, git_cookie):
  # 
  whole_v = whole_v
  list_git = []
  if name_value:
    # print name_value
    for i in name_value:
      # print i
      if i:
        if type(i) is list:
          tmp_index = whole_v.index(i)+1
          return git_callback(whole_v, whole_v[whole_v.index(i)], whole_v[tmp_index:], git_cookie)
        else:
          git_cookie = str(git_cookie.split()[0])+' '+str(git_cookie.split()[1])
          list_git.append(tuple(git_cookie.split("?")+i.split()))
          print list_git
          pool = ThreadPool(100)
          result = pool.map(pool_git, list_git)
          print result
          pool.close()
          pool.join()          
  else:
    print proj_value

The above multi-threaded code snippet is a callback function, not completely according to demo modification, with demo according to the needs of it is not difficult to change, more debugging can be.

python multithreading accepts multiple parameters

from multiprocessing.dummy import Pool as ThreadPool
pool = ThreadPool(100)
result = pool.map(pool_git, list_git)
print result
pool.close()
pool.join()

pool_git is the function you want to call with multiple threads, list_git is the argument that the pool_git function needs to receive, and by default pool_git is a function that receives one argument.

However, our functionality is often designed with complex logic and requires passing in multiple parameters in pool_git, in which case list_git should give a list of multiple tuples.

stackoverflow stackoverflow stackoverflow stackoverflow

def multi_run_wrapper(args):
  return add(*args)
def add(x,y):
  return x+y
if __name__ == "__main__":
  from multiprocessing import Pool
  pool = Pool(4)
  results = pool.map(multi_run_wrapper,[(1,2),(2,3),(3,4)])
  print results
output
[3, 5, 7]

More questions on Stack Overflow for your better understanding:

https://stackoverflow.com/questions/5442910/python-multiprocessing-pool-map-for-multiple-arguments

Believe that smart you 1 can understand ~

Multithreading and multi-process

from multiprocessing.dummy import Pool as ThreadPool

Multithreaded process pool, binding 1 CPU core

from multiprocessing import Pool

Multiprocess, running on multiple cpu cores

If you can't figure out whether CPU is a very intensive task or IO is a very intensive task, use the library to write both import and then instantiate each one for 1 run to see how long it will take. Just swap Pool and ThreadPool by changing a few letters on the creation object.

conclusion