Deep understanding of python multithreaded programming

Process

Concept of process:
Process is the smallest unit of resource allocation, and it is the basic unit of operating system for resource allocation and scheduling. Popular understanding: A running program is a process. For example, qq, wechat, etc. are running, which are all 1 process.
Process Creation Steps
1. Import the process package
import multiprocessing
2. Create process objects through process classes
Process Object = multiprocessing. Process ()
3. Start the process to perform the task
Process Object. start ()

import multiprocessing
import time

def sing():
    for i in range(3):
        print(" Sing. . . ")
        time.sleep(0.5)
def dance():
    for i in range(3):
        print(" Dance. . . ")
        time.sleep(0.5)
if __name__ == '__main__':
    time1 = time.time()
    s1 = multiprocessing.Process(target=sing)
    d1 = multiprocessing.Process(target=dance)
    s1.start()
    d1.start()
    s1.join() # This method can wait for the child process to finish before continuing to run, which is usually used for synchronization between processes 
    d1.join()
    print(time.time()-time1)

Processes with parameters

args tuple, kwargs dictionary

import multiprocessing
import time

def sing(name, num):
    for i in range(num):
        print("%s Singing. . . "%name)
        time.sleep(0.5)
def dance(num):
    for i in range(num):
        print(" Dance. . . ")
        time.sleep(0.5)
if __name__ == '__main__':
    #  Pass parameters in tuples 
    s1 = multiprocessing.Process(target=sing, args=(' Xiao Ming ', 3))
    #  Transmit parameters in the form of dictionaries 
    d1 = multiprocessing.Process(target=dance, kwargs={"num": 5, })
    s1.start()
    d1.start()

Attention points of process

The main process will wait for all child processes to finish execution at the end

Set up the daemon master process

 After the main process ends, it will not continue to execute the remaining work in the child process 
work_process = multiprocessing.Process(target=work, daemon=True)

Thread

Introduction to Threads

Another form of multitasking
Thread is the smallest unit of program execution
Multiple threads belonging to the same process share all the resources owned by the process

Steps to create a thread
Import thread package
import threading
Create a process object through a thread class
Thread Object = threading. Thread ()
Start a thread to execute a task
Thread Object. start ()

Comparison of advantages and disadvantages

1. Advantages and disadvantages of the process:
Advantages: Multi-core can be used and parallel can be used
Disadvantages: High resource overhead
2. Thread advantages and disadvantages:
Advantages: Low resource overhead
Disadvantages: Unavailable multi-core, attached to process 1 process 1 core

Case-Multi-process implementation of video folder multi-task copier

Requirements analysis:
Whether the destination folder exists, if it does not exist, it will be created, and if it exists, it will not be created
Traverse all files in the source folder and copy to the destination folder
Adopt process to realize multitasking and complete copying
Implementation steps
Define the path of the source folder and the path of the destination folder
Create a destination folder
Get the list of files in the source directory through os. listdir
Traverse each file, define a function, and specially realize file copy
Adopt process to realize multi-task and complete high concurrent copy

import os
import multiprocessing
def copy_file(file_name, source_dir, dest_dir):
    # 1  Splice source file path and destination file path 
    source_path = source_dir + '\\' + file_name
    dest_path = dest_dir + '\\' + file_name
    # 2  Open source and destination files 
    with open(source_path, 'rb') as source_file:
        with open(dest_path, 'wb') as dest_file:
            # 3  Loop to read the source file to the destination path 
            while True:
                data = source_file.read(1024)
                if data:
                    dest_file.write(data)
                else:
                    break
if __name__ == '__main__':
    # 1  Define source and destination folders 
    source_dir = r'E:\TCT\TIFF_tran\pos_1'
    dest_dir= r'F:\ Destination folder '

    # 2. Create a destination folder 
    try:
        os.mkdir(dest_dir)
    except:
        print(" Destination folder already exists ")
    # 3. Read the list of files from the source folder 
    file_list = os.listdir(source_dir)
    # 4. Traverse the file list to realize copy 
    for file_name in file_list:
        # copy_file(file_name, source_dir, dest_dir)
        # 5. Using multi-process to realize multi-task copy 
        sub_process = multiprocessing.Process(target=copy_file, args=(file_name, source_dir, dest_dir))
        sub_process.start()

# Thread 
import os
import threading
def copy_file(file_name, source_dir, dest_dir):
    # 1  Splice source file path and destination file path 
    source_path = source_dir + '\\' + file_name
    dest_path = dest_dir + '\\' + file_name
    # 2  Open source and destination files 
    with open(source_path, 'rb') as source_file:
        with open(dest_path, 'wb') as dest_file:
            # 3  Loop to read the source file to the destination path 
            while True:
                data = source_file.read(1024)
                if data:
                    dest_file.write(data)
                else:
                    break
if __name__ == '__main__':
    # 1  Define source and destination folders 
    source_dir = r'F:\ Thunder download \ Video - Intelligent robot from 0 To 1 System introductory course \ Video '
    dest_dir= r'F:\ Destination folder '

    # 2. Create a destination folder 
    try:
        os.mkdir(dest_dir)
    except:
        print(" Destination folder already exists ")
    # 3. Read the list of files from the source folder 
    file_list = os.listdir(source_dir)
    # 4. Traverse the file list to realize copy 
    for file_name in file_list:
        # copy_file(file_name, source_dir, dest_dir)
        # 5. Using multithreading to realize multitask copy 
        sub_thread = threading.Thread(target=copy_file, args=(file_name, source_dir, dest_dir))
        sub_thread.start()
   for file_name in file_list:
       sub_thread.join()
        sub_thread.join()# The main thread waits for all threads to end