Explain the use of the with statement in Python

The introduction

The with statement is a function related to exception handling that is introduced starting with Python 2.5 (with version 2.5 having to be imported via from with 6en__ with_statement) and is available by default starting with version 2.6 (see What's new in Python 2.6? In the with statement section). The with statement is appropriate for accessing resources, ensuring that necessary "cleanup" operations are performed regardless of whether exceptions occur during use, freeing resources such as automatic closing of files after use, automatic acquisition and release of locks in threads, and so on.

The term

To use the with statement, first understand the context manager 1 concept. With the context manager, with statements work.

The following is a set of concepts related to the context manager and with statements.

The context management protocol (Context Management Protocol) : contains methods of s 31en__ () and s 32en__ (), supported

The object of the protocol implements both methods.

Context manager (Context Manager) : an object that supports the context management protocol, which is implemented

S 41en__ () and s 42en__ () methods. The context manager defines the runtime context to be created when an with statement is executed,

Responsible for performing entry and exit operations in the with block context. The context manager is usually invoked using the with statement,

It can also be used by calling its methods directly.

Runtime context (runtime context) : created by the context manager, with the context manager s s 54en__ () and

S 57en__ () method is implemented, s 58en__ () method enters the runtime context before the body of the sentence is executed, s 59en__ () method is implemented

The body of the statement exits from the runtime context after execution. The with statement supports the 1 concept of runtime context.

Context expression (Context Expression) : the expression in the with statement that follows the keyword with

To return a context manager object.

Statement body (with-body) : a code block wrapped around an with statement that calls the context tube before executing the statement body

S 78en__ () method of the processor. After the body of the sentence is executed, s 79en__ () method will be executed.

Basic grammar and working principles

The syntax of the with statement is as follows:
Listing 1. Syntax format of the with statement

  with context_expression [as target(s)]:
    with-body

Here context_expression returns a context manager object, which is not assigned to target(s) in the as clause, but to target(s) with the return value of the context manager's s 96en__ () method, if the as clause is specified. target(s) can be a single variable, or a tuple surrounded by "()" (not just a list of variables separated by ", "but must be" () ")).

Python improves on some of the built-in objects by adding support for a context manager that can be used in with statements, such as the ability to automatically close files, the automatic acquisition and release of thread locks, and so on. If you want to manipulate a file, you can use the with statement to have the following code:
Listing 2. Manipulate the file object with the with statement

  with open(r'somefileName') as somefile:
    for line in somefile:
      print line
      # ...more code

The with statement is used to ensure that the open file handle is closed after the with statement is executed, regardless of whether an exception occurs during file processing. If you use the traditional try/finally paradigm, you use something like this:
Listing 3. try/finally way to manipulate the file object

  somefile = open(r'somefileName')
  try:
    for line in somefile:
      print line
      # ...more code
  finally:
    somefile.close()

In comparison, using the with statement can reduce the amount of encoding. Support for context management protocols has been added to modules threading, decimal, and so on.

PEP 0343 describes the implementation of the with statement. The execution of the with statement is similar to the following code block:
Listing 4. with statement execution procedure

       

context_manager = context_expression
  exit = type(context_manager).__exit__ 
  value = type(context_manager).__enter__(context_manager)
  exc = True  # True  Represents normal execution, even if there is an exception is ignored; False  Represents a rethrow of an exception that needs to be handled 
  try:
    try:
      target = value #  If you use  as  clause 
      with-body   #  perform  with-body
    except:
      #  An exception occurred during execution 
      exc = False
      #  if  __exit__  return  True , the exception is ignored; If the return  False , then rethrow the exception 
      #  The exception is handled by the outer code 
      if not exit(context_manager, *sys.exc_info()):
        raise
  finally:
    #  Normal exit, or pass  statement-body  In the  break/continue/return  Statement out 
    #  Or ignore the exception to exit 
    if exc:
      exit(context_manager, None, None, None) 
    #  The default return  None . None  Think of it in the Boolean context  False

      executes context_expression, generating the context manager context_manager
      calls the context manager's s 148en__ () method; If the as clause is used, assign the return value of the method s 150en__ () to target(s) in the as clause
      with body
Execute the context manager's s s 164en__ () method, s 165en__ () method, which is responsible for performing "clean up" tasks, such as freeing resources, regardless of whether or not an exception has occurred during the execution. If there is no exception during the execution, or if the statement break/continue/return is executed in the body of the statement, then call s 170en__ (None, None, None) with s 169en as the parameter. If an exception occurs during execution, the exception information obtained using sys.exc_info is used to call s 177en__ (exc_type, exc_value, exc_traceback)
If s 185en     returns False (type, value, traceback) with an exception, then the exception will be thrown again, allowing the statement logic other than with to handle the exception, which is also the general practice. If True is returned, the exception is ignored and no longer handled

Customize the context manager

Developers can customize the classes that support the context management protocol. Custom context manager with two methods, s/s 199en__ () and s/s 200en__ (), required to implement the context management protocol:

S 203en     context_manager. s 208en__ () : enter the runtime context of the context manager and call it before the body of the statement is executed. The with statement assigns the return value of this method to target in the as clause, if the as clause is specified
S 214en   manager.s 219en__ (exc_type, exc_value, exc_traceback) : exits the runtime context associated with the context manager and returns a Boolean value indicating whether an exception has been handled or not. The parameter represents the exception that causes the exit operation. If no exception occurs at exit, all three parameters are None. If an exception occurs, return

      True means no exception is handled, otherwise the exception is rethrown after exiting the method to be handled by code logic outside of the with statement. If an exception is generated inside this method, it replaces the exception generated by the statement in statement-body. When handling an exception, do not display a rethrow exception, that is, an exception that cannot be rethrown if passed in as a parameter, just set the return value to False. The context management code then detects whether s 237en__ () fails to handle the exception

Here is a simple example to show you how to build a custom context manager. Note that the context manager must provide definitions of both s 240en__ () and s 241en__ () methods. The absence of either will result in s 242en; The with statement will first check to see if the s 244en__ () method is provided, and then to see if the s 245en__ () method is defined.

Suppose you have a resource, DummyResource, which needs to be allocated before access and released after use. The distribution operation can be placed into the method of s 249en__ (), and the release operation can be placed into the method of s 250en__ (). For the sake of simplicity, there is no actual resource allocation and release, but only print statements to indicate the current operation.
Listing 5. Customize the object that supports the with statement

  class DummyResource:
  def __init__(self, tag):
      self.tag = tag
      print 'Resource [%s]' % tag
    def __enter__(self):
      print '[Enter %s]: Allocate resource.' % self.tag
      return self  #  You can return different objects 
    def __exit__(self, exc_type, exc_value, exc_tb):
      print '[Exit %s]: Free resource.' % self.tag
      if exc_tb is None:
        print '[Exit %s]: Exited without exception.' % self.tag
      else:
        print '[Exit %s]: Exited with exception raised.' % self.tag
        return False  #  Can be omitted, default None It's also seen as False

S 257en__ () in DummyResource returns its own reference, which can be assigned to the target variable in as clause; The type of the return value can be set to a different type depending on actual needs, not the context manager object itself.

The variable exc_tb is detected in the method of s262en__ (). If it is not None, it means that an exception has occurred. Returning False means that the exception needs to be handled by the external code logic. Note that if there is no exception, the default return value is None, which is also considered as False in a Boolean environment. However, as there is no exception, the three parameters of s 269en__ () are all None. The context management code can detect this and do the normal processing.

Now access DummyResource in the with statement:
Listing 6. Object with custom with support statement

     

 with DummyResource('Normal'):
    print '[with-body] Run without exceptions.'

  with DummyResource('With-Exception'):
    print '[with-body] Run with exception.'
    raise Exception
    print '[with-body] Run with exception. Failed to finish statement-body!'

The results of the first with statement are as follows:
Listing 7. with statement 1 executes the result

      Resource [Normal]
      [Enter Normal]: Allocate resource.
      [with-body] Run without exceptions.
      [Exit Normal]: Free resource.
      [Exit Normal]: Exited without exception.

As you can see, during normal execution, the body with-body will be finished first, and then the method of s 301en__ () will be executed to release the resources.

The results of the second with statement are as follows:
Listing 8. with statement 2 executes the result

   

 Resource [With-Exception]
  [Enter With-Exception]: Allocate resource.
  [with-body] Run with exception.
  [Exit With-Exception]: Free resource.
  [Exit With-Exception]: Exited with exception raised.

  Traceback (most recent call last):
   File "G:/demo", line 20, in <module>
    raise Exception
  Exception

As you can see, when an exception occurs in with-body, with-body does not execute completely, but the resource is guaranteed to be freed, and the generated exception is captured by the code logic outside the with statement.

You can customize the context manager to manage resources in the software system, such as database connections, access control for Shared resources, and so on. The Python online document Writing Context Managers provides a simple example of a context manager for managing database connections.

contextlib module

The contextlib module provides three objects: decorator contextmanager, function nested, and context manager closing. Using these objects, you can wrap existing generator functions or objects, add support for the context management protocol, and avoid having to write a context manager specifically to support with statements.
A decorator contextmanager

contextmanager is used to decorate the generator function. After the generator function is decorated, a context manager is returned with its s 339en__ () and s 340en__ () methods provided by contextmanager instead of the previous iteration. The decorated generator function can only produce one value, otherwise it will result in an exception RuntimeError; The resulting value is assigned to target in the as clause, if the as clause is used. Let's look at a simple example.
Listing 9. Example of using the decorator contextmanager

     

from contextlib import contextmanager

  @contextmanager
  def demo():
    print '[Allocate resources]'
    print 'Code before yield-statement executes in __enter__'
    yield '*** contextmanager demo ***'
    print 'Code after yield-statement executes in __exit__'
    print '[Free resources]'

  with demo() as value:
    print 'Assigned Value: %s' % value

The output is as follows:
Listing 10. contextmanager USES the example to execute the results

   

 [Allocate resources]
  Code before yield-statement executes in __enter__
  Assigned Value: *** contextmanager demo ***
  Code after yield-statement executes in __exit__
  [Free resources]

As you can see, the statements before yield in the generator function are executed in the method of s 363en__ (), the ones after s 364en are executed in s 365en__ (), and the values generated by s 366en are assigned to the s 368en variable in s 367en clause.

It should be noted that contextmanager only omits the writing of s 372en__ ()/s 373en__ (), but is not responsible for the "fetch" and "clean up" of the resources; "Get" action needs to be defined before yield statements, "clean" need to define yield statement after the operation, such with statements in performing __enter__ () / __exit__ () method will perform these statements in order to obtain/release resources, namely generator need to implement the necessary logic control functions, including resource access an appropriate exception when there is an error.
Function nested

nested can organize multiple context managers in 1, avoiding the use of nested with statements.
Listing 11. nested syntax

     

 with nested(A(), B(), C()) as (X, Y, Z):
     # with-body code here

Similar to:
Listing 12. nested executes the procedure

  with open(r'somefileName') as somefile:
    for line in somefile:
      print line
      # ...more code

0

It is important to note that if a context manager's s method of s 398en__ () returns False for exception handling after an exception has occurred, the outer context manager will not detect the exception.
Context manager closing

The implementation of closing is as follows:
Listing 13. Context management closing implementation

 

  with open(r'somefileName') as somefile:
    for line in somefile:
      print line
      # ...more code

1

The context manager assigns the wrapped object to the target variable of the as clause, and ensures that the open object is closed when with-body is finished executing. The object wrapped by the closing context manager must provide the definition of the close() method, otherwise an AttributeError error will be reported on execution.
Listing 14. Custom objects that support closing

     

  with open(r'somefileName') as somefile:
    for line in somefile:
      print line
      # ...more code

2

The output is as follows:
Listing 15. The output of the custom closing object

  Acquire resources.
  Using resources
  Clean up any resources acquired.

closing is suitable for objects that provide an close() implementation, such as network connections, database connections, and so on, as well as for performing the required "cleanup" of resources through the interface close() when customizing classes.