The with keyword in Python is explained in detail

In Python 2.5, the with keyword is added. It will be commonly used by try... except... finally... Patterns are easily reusable. Here's a classic example:

with open('file.txt') as f:
  content = f.read()

In this code, no matter what happens to the code block in with during execution, the file will eventually be closed. If an exception occurs during the execution of a block of code, the program will close the open file before the exception is thrown.

Let's do another example.

When initiating a database transaction request, it is common to use code like this:

db.begin()

try:
  # do some actions
except:
  db.rollback()
  raise
finally:
  db.commit()

If you make the operation that initiated the transaction request to support the with keyword, you can use code like this:

with transaction(db):
  # do some actions

Below, I will explain the execution of with in detail and implement the above code in two common ways.

The 1 - like execution of with

1. The basic with expression is as follows:

with EXPR as VAR:
  BLOCK

Where: EXPR can be any expression; as VAR is optional. The 1-like execution process is as follows:

Calculate EXPR and get a context manager. The context manager's s s 36en () method is saved for subsequent calls. Invoke the context manager's s 37en () method. If the with expression contains as VAR, the return value of EXPR is assigned to VAR. Execute the expression in BLOCK. Invoke the context manager's s 44en () method. If an exception occurs during the execution of BLOCK and causes the program to exit, the return values of type, value and traceback (i.e. sys.exc_info ()) of the exception will be passed to the method of s 52en (). Otherwise, three None are passed.

This process is represented in code as follows:

mgr = (EXPR)
exit = type(mgr).__exit__ #  There is no implementation 
value = type(mgr).__enter__(mgr)
exc = True

try:
  try:
    VAR = value #  If you have  as VAR
    BLOCK
  except:
    exc = False
    if not exit(mgr, *sys.exc_info()):
      raise
finally:
  if exc:
    exit(mgr, None, None, None)

There are several details to this process:

If there is no one method in the context manager, s 61en () or s 62en (), then the interpreter will throw 1 AttributeError.
If, after an exception has occurred in BLOCK, the method returns a value that can be considered as True, the exception will not be thrown and the following code will continue.

So let's do this in two ways.

Implement the context manager class

The first method is to implement a class with one instance property, db (), and the methods required by the context manager, s/s 76en () and s/s 77en ().

class transaction(object):
  def __init__(self, db):
    self.db = db

  def __enter__(self):
    self.db.begin()

  def __exit__(self, type, value, traceback):
    if type is None:
      db.commit()
    else:
      db.rollback()

After understanding the implementation of with, this implementation is easy to understand. The following implementation is much more complex to understand.

Use a generator decorator

In the standard library of Python, there is a decorator that can get the context manager through the generator. The implementation using the generator decorator is as follows:

from contextlib import contextmanager

@contextmanager
def transaction(db):
  db.begin()

  try:
    yield db
  except:
    db.rollback()
    raise
  else:
    db.commit()

On the first eye, this implementation is simpler, but the mechanism is more complex. Here's how it works:

When the Python interpreter recognizes the yield keyword, def creates a generator function instead of a regular function (I like to use functions instead of methods outside of class definitions). The decorator contextmanager is called and returns a help method, which generates an instance of GeneratorContextManager when called. The EXPR in the final with expression calls the help function returned by the contentmanager decorator. The with expression calls transaction(db), which is actually a call to the help function. The help function calls the generator function, and the generator function creates a generator. The help function passes the generator to GeneratorContextManager and creates an instance object of GeneratorContextManager as the context manager. The with expression calls the context manager's s 105en () method of the instance object. The next() method of this generator will be called in the method s. At this point, the generator method will stop at yield db and use db as the return value of next(). If you have as VAR, then it will be assigned to VAR. BLOCK in with is executed. After the completion of BLOCK, invoke the context manager's s 118en () method. S 119en () method calls the generator's next() method again. If an StopIteration exception occurs, pass. If no exception occurs, the generator method will execute db.commit (), otherwise db.rollback ().

Take a look again at the code for the above procedure:

def contextmanager(func):
  def helper(*args, **kwargs):
    return GeneratorContextManager(func(*args, **kwargs))
  return helper

class GeneratorContextManager(object):
  def __init__(self, gen):
    self.gen = gen

  def __enter__(self):
    try:
      return self.gen.next()
    except StopIteration:
      raise RuntimeError("generator didn't yield")

  def __exit__(self, type, value, traceback):
    if type is None:
      try:
        self.gen.next()
      except StopIteration:
        pass
      else:
        raise RuntimeError("generator didn't stop")
    else:
      try:
        self.gen.throw(type, value, traceback)
        raise RuntimeError("generator didn't stop after throw()")
      except StopIteration:
        return True
      except:
        if sys.exc_info()[1] is not value:
          raise

conclusion

The with expression of Python contains many Python features. It is well worth the time to get through with.

1 some other examples

Locking mechanism

@contextmanager
def locked(lock):
  lock.acquired()
  try:
    yield
  finally:
    lock.release()

Standard output redirection

@contextmanager
def stdout_redirect(new_stdout):
  old_stdout = sys.stdout
  sys.stdout = new_stdout
  try:
    yield
  finally:
    sys.stdout = old_stdout

with open("file.txt", "w") as f:
  with stdout_redirect(f):
    print "hello world"

The resources

The Python "with" Statement by Example

PEP 343