Python multithreading usage example
- 2020-04-02 14:28:32
- OfStack
This article gives an example of python multithreading usage. Share with you for your reference. The details are as follows:
Today, when I was trying to learn python multi-threading, I suddenly found that I was not very clear about the use of super, so I will summarize some problems encountered. When I tried to write the following code:
class A():
def __init__( self ):
print "A"
class B( A ):
def __init__( self ):
super( B, self ).__init__( )
# A.__init__( self )
print "B"
b = B()A:
The self, super (B) __init__ ()
TypeError: must be type, not classobj
It turned out to be A problem with new classes in python, which meant that A had to be new. The solutions are as follows:
(1)
class A( object ):
def __init__( self ):
print "A"
class B( A ):
def __init__( self ):
super( B, self ).__init__( )
# A.__init__( self ) ## This statement is old-fashioned and potentially problematic and should be avoided
print "B"
b = B()
(2)
__metaclass__=type
class A():
def __init__( self ):
print "A"
class B( A ):
def __init__( self ):
super( B, self ).__init__( )
# A.__init__( self ) ## This statement is old-fashioned and potentially problematic and should be avoided
print "B"
b = B()Pay attention to : if you are in super(B, self). )
S, s, s, s, s, s, s, s, s, s, s, s, s, s, s, s, s.
S (B, self). S (self).
TypeError: s/s () takes exactly 1 argument (2 given)
The above is just a little my experience notes, ha ha.
import threading, time
class myThread( threading.Thread ):
def __init__( self, threadname = "" ):
#threading.Thread.__init__( self, name = threadname )
super( myThread, self ).__init__( name = threadname )
def run( self ):
print "starting====", self.name, time.ctime()
time.sleep( 5 )
print "end====", self.name, time.ctime(),
m = myThread( "m" )
n = myThread( "n" )
m.start()
n.start()Output results:
Starting ==== m Mon Aug 08 21:55:41 2011
Starting ==== n Mon Aug 08 21:55:41 2011
If the main thread of a process runs out and the children are still executing, the process does not exit until all the children have finished. Here's an example:
import threading, time
class myThread( threading.Thread ):
def __init__( self, threadname = "" ):
#threading.Thread.__init__( self, name = threadname )
super( myThread, self ).__init__( name = threadname )
def run( self ):
print "starting====", self.name, time.ctime()
time.sleep( 5 )
print "end====", self.name, time.ctime(),
m = myThread( "m" )
m.start()
print "main end"
printThe output result is:
Starting ==== m Mon Aug 08 22:01:06 2011
The main end
End ==== m Mon Aug 08 22:01:11 2011
After the main process has finished, the child process has not finished
If we want the child to end at the end of the main process, we should use the setDaemon () function.
Examples are as follows:
import threading, time
class myThread( threading.Thread ):
def __init__( self, threadname = "" ):
#threading.Thread.__init__( self, name = threadname )
super( myThread, self ).__init__( name = threadname )
def run( self ):
print "starting====", self.name, time.ctime()
time.sleep( 5 )
print "end====", self.name, time.ctime(),
m = myThread( "m" )
m.setDaemon( True )
m.start()
print "main end"
printThe output result is: starting====main end m Mon Aug 08 22:02:58 2011
As you can see, the "end===..." that should have been printed at the end of the child process m is not printed.
Simple thread synchronization
Multiple threads of execution often share data, which is fine if you just read the Shared data, but can have unexpected results if multiple threads modify the Shared data.
If two thread objects, t1 and t2, were to increment num=0 by 1, then t1 and t2 would each modify num 10 times, so the final result of num would be 20. But if, when t1 gets the value of num (if num is 0), the system dispatches t1 as "sleeping", and then t2 transitions to "running", then t2 gets the value of num as 0, and then he assigns the value of num+1 to num. The system then converts t2 to the state of "sleeping" and t1 to the state of "running". Since t1 has got the value of num to be 0, he also assigns the value of num+1 to num to be 1. It was run 2 times by 1, but num was only increased 1 time. Something like this is possible when multiple threads execute simultaneously. So in order to prevent this kind of situation we need to use the thread synchronization mechanism.
The simplest synchronization mechanism is "locking"
The lock object is created with the threading.RLock class
mylock = threading.RLock()
How do you use locks to synchronize threads? A thread can use the acquire() method of the lock so that the lock enters the "locked" state. Only one thread at a time can acquire the lock. If another thread when trying to get the lock will be system becomes "blocked" status, until the thread calls the lock with lock release () method (release), the lock will enter the "unlocked" state. The thread in the "blocked" state receives a notification and has the right to acquire the lock. If multiple threads are in the "blocked" state, all threads will first lift the "blocked" state, and then the system selects one thread to acquire the lock, while other threads remain silent (" blocked ").
import threading
mylock = threading.RLock()
class mythread(threading.Thread)
...
def run(self ...):
... # here Can not be Place code that modifies Shared data
mylock.acquire()
... # here can Place code that modifies Shared data
mylock.release()
... # here Can not be Place code that modifies Shared data We refer to the code that modifies the Shared data as a "critical section", all of which must be enclosed between the acquire() and release() method calls of the same lock object.
Locks provide only the most basic level of synchronization. Sometimes more complex thread synchronization is required, such as accessing a critical section only when certain events occur (such as when a certain value changes). This requires the use of "conditional variables".
The Condition variable is created with the threading.Condition class
mycondition = threading.Condition()How do conditional variables work? First, after a thread successfully obtains a condition variable, calling the wait() method of the condition variable causes the thread to release the lock and enter the "blocked" state, until another thread invokes the notify() method of the same condition variable to wake up the "blocked" thread. Calling the notifyAll() method of the condition variable wakes up all waiting threads.
A deadlock occurs when a program or thread is permanently blocked. So if you use synchronization mechanisms such as locks, condition variables, and so on, be sure to check carefully to prevent deadlock situations. The finally clause in the exception-handling mechanism is used to ensure the release of the lock for critical sections where exceptions may occur. A thread waiting for a condition variable must explicitly wake up with the notify() method or remain silent forever. Make sure that every call to the wait() method has a corresponding call to notify(), and of course, call the notifyAll() method just in case.
Synchronous queue
We often use two threads of the producer/consumer relationship to process data from a Shared buffer. For example, a producer thread accepts user data into a Shared buffer and waits for a consumer thread to fetch the data. But if the buffer is too small and the speed of the producer and consumer asynchronous threads is not the same, it is easy for one thread to wait for the other. To minimize the wait time for threads that share resources and work at the same speed, we can use a "queue" to provide additional buffers.
To create a "queue" object, you can use the following code:
import Queue
myqueue = Queue.Queue(maxsize = 10)Queue.Queue class is a synchronous implementation of a Queue. The queue length can be infinite or finite. The Queue length can be set by maxsize, an optional parameter to the constructor of the Queue. If the maxsize is less than 1, the queue length is infinite.
Queue a value:
Myqueue. Put (10)
The call to the queue object's put() method inserts an item at the end of the queue. Put () has two parameters, the first item is required, is the value of the inserted item; The second block is optional and defaults to 1. If the queue is currently empty and the block is 1, the put() method suspends the calling thread until a data cell is vacated. If the block is 0, the put method throws a Full exception.
To remove a value from the queue:
Myqueue. The get ()
Call the get() method of the queue object to remove from the queue header and return an item. The optional parameter is block, and the default is 1. If the queue is empty and the block is 1, get() suspends the calling thread until an item is available. If the block is 0, the queue throws an Empty exception.
Let's use an example to show how to use Queue:
# queue_example.py
from Queue import Queue
import threading
import random
import time
# Producer thread
class Producer( threading.Thread ):
def __init__( self, threadname, queue ):
threading.Thread.__init__( self, name = threadname )
self.sharedata = queue
def run( self ):
for i in range( 20 ):
print self.getName(), 'adding', i, 'to queue'
self.sharedata.put( i )
time.sleep( random.randrange( 10 ) / 10.0 )
print self.getName(), 'Finished'
# Consumer thread
class Consumer( threading.Thread ):
def __init__( self, threadname, queue ):
threading.Thread.__init__( self, name = threadname )
self.sharedata = queue
def run( self ):
for i in range( 20 ):
print self.getName(), 'got a value:', self.sharedata.get()
time.sleep( random.randrange( 10 ) / 10.0 )
print self.getName(), 'Finished'
# Main thread
def main():
queue = Queue()
producer = Producer( 'Producer', queue )
consumer = Consumer( 'Consumer', queue )
print 'Starting threads ...'
producer.start()
consumer.start()
producer.join()
consumer.join()
print 'All threads have terminated.'
if __name__ == '__main__':
main()The output result of the program is:
Starting threads...
The Producer adds 0 to queue
Consumer got a value: 0
Producer Finished
The Producer draws 1 to queue
Producer Finished
The Producer draws 2 to queue
Consumer Finished
Consumer got a value: 1
Consumer Finished
Consumer got a value: 2
Consumer Finished
Consumer got a value: Producer Finished
The Producer draws 3 to queue
3
Consumer Finished
Consumer got a value: Producer Finished
The Producer adding 4 to queue
4
ConsumerProducer Finished
ConsumerFinished
Got a value: the Producer draws 5 to queue
5
Consumer Finished
Consumer got a value: Producer Finished
The Producer draws 6 to queue
Producer Finished
The Producer draws 7 to queue
6
Consumer Finished
Consumer got a value: 7
Producer Finished
The Producer draws 8 to queue
Producer Finished
The Producer draws 9 to queue
Consumer Finished
Consumer got a value: 8
ConsumerProducer FinishedFinished
ConsumerProducer Got a value: adding 109
To the queue
Producer Finished
The Producer draws 11 to queue
Producer Finished
The Producer adding 12 to queue
ConsumerProducer FinishedFinished
ConsumerProducer Got a value: adding 1310
To the queue
Producer Finished
The Producer adding 14 to queue
Consumer Finished
Consumer got a value: 11
Producer Finished
The Producer draws 15 to queue
Producer Finished
The Producer draws 16 to queue
Producer Finished
The Producer draws 17 to queue
Producer Finished
The Producer adding 18 to queue
Consumer Finished
Consumer got a value: 12
Producer Finished
The Producer adding 19 to queue
Producer Finished
Consumer Finished
Consumer got a value: 13
Consumer Finished
Consumer got a value: 14
Consumer Finished
Consumer got a value: 15
Consumer Finished
The Consumer got a value: 16
Consumer Finished
Consumer got a value: 17
Consumer Finished
Consumer got a value: 18
Consumer Finished
Consumer got a value: 19
Consumer Finished
All threads have terminated.
I hope this article has helped you with your Python programming.