Use Python to implement read write lock sample code

start

The multithreading model provided by Python does not provide read-write locks. Compared with simple mutex locks, read-write locks have higher applicability. Multiple threads can simultaneously occupy read-write lock in read mode, but only one thread can occupy read-write lock in write mode.

When there is no write lock, a read lock can be added and any thread can add it at the same time. A write lock can only have one thread and must be added when there is no read lock.

Simple implementation

import threading

class RWlock(object):
  def __init__(self):
    self._lock = threading.Lock()
    self._extra = threading.Lock()
    self.read_num = 0

  def read_acquire(self):
    with self._extra:
      self.read_num += 1
      if self.read_num == 1:
        self._lock.acquire()

  def read_release(self):
    with self._extra:
      self.read_num -= 1
      if self.read_num == 0:
        self._lock.release()

  def write_acquire(self):
    self._lock.acquire()

  def write_release(self):
    self._lock.release()

self.read_num This is a simple implementation of the read/write lock. self.read_num is used to store the number of threads that have acquired the read lock. This property belongs to the critical section.

But the lock is unfair. Ideally, a thread should have the same chance of getting a read or a write operation. As you can see from the above code, the read request immediately sets self.read_num += 1, whether or not the lock is acquired, while the write request waits for read_num to obtain the lock.

As a result, write permission can be obtained only if the lock is not held or if no read request is made. We should try to avoid reading mode locks for long periods of time.

Priority of read/write locks

Read/write locks also have read/write priority. The above code is read first.

If you want to change it to write first, then change it to record the reference count of the writer thread. If the read and write are competing at the same time, you can ask the writer thread to increase the count of the write, so that the reader thread can not get the read lock 1, because the reader thread must first determine the write reference count, if not 0, wait for it to be 0, and then read. This part of the code will not be listed.

But that's not flexible enough. We don't need two similar read-write lock classes. We want to refactor our code to make it more powerful.

To improve the

In order to satisfy a custom priority read-write lock, the number of waiting read-write threads is recorded, and two conditions, threading.Condition, are required to handle the notification of which side has priority. Counting references can amplify semantics: positive: number of threads that are reading, negative: number of threads that are writing (up to -1)

When obtaining a read operation, first judge that there is a waiting writer thread, if there is no, proceed to read operation, if there is, wait for the count of read to be increased by 1, wait for Condition notification; If the condition is not true, the loop waits; if the condition is not true, the loop waits.

When a write is obtained, if the lock is not held, the reference count is decreased by 1. If the lock is held, the number of writer threads is increased by 1 and the write condition Condition is notified.

The release of read mode and write mode is the same, so the corresponding Condition should be notified according to the judgment:

class RWLock(object):
  def __init__(self):
    self.lock = threading.Lock()
    self.rcond = threading.Condition(self.lock)
    self.wcond = threading.Condition(self.lock)
    self.read_waiter = 0  #  Number of threads waiting to acquire a read lock 
    self.write_waiter = 0  #  Number of threads waiting to acquire a write lock 
    self.state = 0     #  Positive: Indicates the number of threads being read    Negative: Indicates the number of threads that are writing (maximum) -1 ) 
    self.owners = []    #  The thread being operated on id A collection of 
    self.write_first = True #  Default write first, False Denote read priority 

  def write_acquire(self, blocking=True):
    #  Obtain a write lock only if 
    me = threading.get_ident()
    with self.lock:
      while not self._write_acquire(me):
        if not blocking:
          return False
        self.write_waiter += 1
        self.wcond.wait()
        self.write_waiter -= 1
    return True

  def _write_acquire(self, me):
    #  A write lock is acquired only when the lock is unopened or when the current thread is already in possession 
    if self.state == 0 or (self.state < 0 and me in self.owners):
      self.state -= 1
      self.owners.append(me)
      return True
    if self.state > 0 and me in self.owners:
      raise RuntimeError('cannot recursively wrlock a rdlocked lock')
    return False

  def read_acquire(self, blocking=True):
    me = threading.get_ident()
    with self.lock:
      while not self._read_acquire(me):
        if not blocking:
          return False
        self.read_waiter += 1
        self.rcond.wait()
        self.read_waiter -= 1
    return True

  def _read_acquire(self, me):
    if self.state < 0:
      #  If the lock is held by a write lock 
      return False

    if not self.write_waiter:
      ok = True
    else:
      ok = me in self.owners
    if ok or not self.write_first:
      self.state += 1
      self.owners.append(me)
      return True
    return False

  def unlock(self):
    me = threading.get_ident()
    with self.lock:
      try:
        self.owners.remove(me)
      except ValueError:
        raise RuntimeError('cannot release un-acquired lock')

      if self.state > 0:
        self.state -= 1
      else:
        self.state += 1
      if not self.state:
        if self.write_waiter and self.write_first:  #  If a write operation is waiting (default write first) 
          self.wcond.notify()
        elif self.read_waiter:
          self.rcond.notify_all()
        elif self.write_waiter:
          self.wcond.notify()

  read_release = unlock
  write_release = unlock