python with early exits encountered with potholes and solutions

Origin of the problem

Earlier, with was used to implement version 1 of the global process lock, hoping to achieve the following effects:

with CacheLock("test_lock", 10):
  # If the lock is preempted, this code is executed 
  #  Otherwise, let with Early exit 

Not to mention the global process lock itself, which mostly relies on external caches, setnx is used on redis, sometimes with cache breakdown issues and random delays as needed to prevent stress on the cache itself

Unit tests were also written to test the validity of this code:

with CacheLock("test_lock", 10):
  value = cache.get("test_lock")
  self.assertEqual(value, 1)
  with CacheLock("test_lock", 10):
    #  It's not going to come in here 
    self.assertFalse(True)
value = cache.get("test_lock")
self.assertEqual(value, None)

It looks like it passed perfectly.

This 1 global process lock is achieved by throwing an exception with the method of ___ 16EN__, and catching an exception with the method of ___ 17en__ :

class CacheLock(object):
  def __init__(self, lock_key, lock_timeout):
    self.lock_key = lock_key
    self.lock_timeout = lock_timeout
    self.success = False
  def __enter__(self):
    self.success = cache.lock(self.lock_key, self.lock_timeout)
    if self.success:
      return self
    else:
      raise LockException("not have lock")
  def __exit__(self, exc_type, exc_value, traceback):
    # Nothing until you grab the lock? 
    if self.success:
      await cache.delete(self.lock_key)
    if isinstance(exc_value, LockException):
      return True
    if exc_type:
      raise exc_value

It looks good, after all the unit tests.

However, such an implementation is problematic:

The reason is that the execution of each ___ 25EN__ is not wrapped up with ___ 26EN__, and therefore the exceptions thrown by each ___ 27EN__ will not be caught by ___.

The above unit test passes precisely because there are two with statements with the outside with catching the exception thrown by the inner ___ enter__

Using improved unit tests:

cache.set("test_lock",1)
with CacheLock("test_lock", 10):
  self.assertFalse(True)
value = cache.get("test_lock")
self.assertEqual(value, None)

You're going to find that the unit tests don't pass.

This problem is that I tried to implement another logic using with: the AB test with the same exception thrown by ___ 43en__, with succexit__ trying to catch:

import operator
class EarlyExit(Exception):
  pass
class ABContext(object):
  """AB Test context 
  >>> with ABContext(newVersion, consts.ABEnum.layer2):
  >>>   # dosomething
  """
  def __init__(self, version, ab_layer, relationship="eq"):
    self.version = version
    self.ab_layer = ab_layer
    #  If no such operator exists, an error is reported in advance 
    self.relationship = getattr(operator, relationship)
  def __enter__(self):
    #  If the condition is not met, the content in the context is not executed 
    if not self.relationship(self.version, self.ab_layer.value):
      raise EarlyExit("not match")
    return self
  def __exit__(self, exc_type, exc_value, traceback):
    if exc_value is None:
      return True
    if isinstance(exc_value, EarlyExit):
      return True
    if exc_type:
      raise exc_value
    return True

Debugging failed unit tests finds that an exception has not been thrown by ___

The first solution

If you've got the execution order of with, the first solution is in order: since the catch exceptions of each ___ 54EN__ are executed, let's provide a function with each ___, and change the implementation of ABContext to this:

def ensure(self):
    if not self.relationship(self.version, self.ab_layer.value):
      raise EarlyExit("not match")
  def __enter__(self):
    #  If the condition is not met, the content in the context is not executed 
    return self

When used:

with ABContext(newVersion, consts.ABEnum.layer2) as c:
  c.ensure()
  #  Execute any other code you want to execute 

However, such a solution is not elegant. If you forget to use ensure when using ABContext, you will not use Context at all. It is too easy to make mistakes, and the code has lost the properties of Pythonic

The second solution

Looking through the standard library documentation of contextlib, I found an obsolete function: ES72en.nested

from contextlib import nested
with nested(*managers):
  do_something()

Multiple contexts can be executed.

from contextlib import nested
with nested(A(), B(), C()) as (X, Y, Z):
  do_something()
# is equivalent to this:
m1, m2, m3 = A(), B(), C()
with m1 as X:
  with m2 as Y:
    with m3 as Z:
      do_something()

After Python2.7, this obsolete feature can be executed directly by the with keyword, as follows:

with context1,context2:
  #do something

Per the order of execution of each ___, we can implement 1 catch with the first context __, and the second context ___ to throw an exception,

It goes like this:

with CacheLock("test_lock", 10):
  value = cache.get("test_lock")
  self.assertEqual(value, 1)
  with CacheLock("test_lock", 10):
    #  It's not going to come in here 
    self.assertFalse(True)
value = cache.get("test_lock")
self.assertEqual(value, None)

0

The use of ABContext combined with the previous implementation is as follows:

with CacheLock("test_lock", 10):
  value = cache.get("test_lock")
  self.assertEqual(value, 1)
  with CacheLock("test_lock", 10):
    #  It's not going to come in here 
    self.assertFalse(True)
value = cache.get("test_lock")
self.assertEqual(value, None)

1

good, that's it for the unit tests

Can you put more pressure on it?

Indeed, to write two context in with is a bit painful and not particularly elegant, but can we go back to the original usage: we only write one context, and the one context does two context things?

If only the nested function were still there. That's what it does.

After Python3.1, contextlib provides the functionality of 1 ExitStack to provide the functionality of 1 simulation, but try 1 and find that it actually calls the method with only a ___ 117EN__ without doing the corresponding exception catch

The third solution

Ha ha ha ha to circle around oneself, want to 1, the same is a code block indentation, why can't use if to solve! Isn't it:

with CacheLock("test_lock", 10):
  value = cache.get("test_lock")
  self.assertEqual(value, 1)
  with CacheLock("test_lock", 10):
    #  It's not going to come in here 
    self.assertFalse(True)
value = cache.get("test_lock")
self.assertEqual(value, None)

2

TIL

In short, I learned some useful functions and decorators in contextlib, and it was the first time that with could put an context

While e the dynamic construction of putting more than one context has yet to be investigated, the block following with cannot fill in a single tuple or list. Disappointed..