An in depth explanation of asyncio module in Python

1. Overview

The asyncio module in Python has built-in support for asynchronous IO to handle asynchronous IO.It is a standard library introduced by Python version 3.4.

The programming model for asyncio is a message loop.We implement asynchronous IO by taking a reference to an EventLoop directly from the asyncio block and throwing the required protocol into EventLoop.

2. Implement Hello world with asyncio

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# @Time : 2019/1/9 11:23
# @Author : Arrow and Bullet
# @FileName: test.py
# @Software: PyCharm
# @Blog  : https://blog.csdn.net/qq_41800366
import asyncio

@asyncio.coroutine
def hello():
 print("Hello world!")
 #  Asynchronous call asyncio.sleep(2): 
 yield from asyncio.sleep(2)
 print("Hello again!")

#  Obtain EventLoop:
loop = asyncio.get_event_loop()
#  implement coroutine
loop.run_until_complete(hello())
loop.close()

@asyncio.coroutine marks an generator as coroutine type, and then we throw the coroutine into EventLoop for execution.

hello() will print out Hello world first!,The yield from syntax then allows us to easily call another generator.Since asyncio.sleep() is also an coroutine, the thread does not wait for asyncio.sleep(), but simply interrupts and executes the next message loop.When asyncio.sleep () returns, the thread gets the return value from yield from (in this case, None), then executes the next line of statements.

Consider asyncio.sleep(2) as a 2-second IO operation (such as reading a large file), during which the main thread does not wait but executes the other coroutines that can be executed in EventLoop, so concurrent execution is possible.

Let's try encapsulating two coroutines with task:

import threading
import asyncio


@asyncio.coroutine
def hello():
 print('Hello world! (%s)' % threading.currentThread())
 yield from asyncio.sleep(2)
 print('Hello again! (%s)' % threading.currentThread())


loop = asyncio.get_event_loop()
tasks = [hello(), hello()]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

Observe the execution process:

Hello world! ( < _MainThread(MainThread, started 140735195337472) > )
Hello world! ( < _MainThread(MainThread, started 140735195337472) > )
(pause for about 2 seconds)
Hello again! ( < _MainThread(MainThread, started 140735195337472) > )
Hello again! ( < _MainThread(MainThread, started 140735195337472) > )

As you can see from the current thread name printed, two coroutines are executed concurrently by the same thread.

If asyncio.sleep() is replaced with a true IO operation, multiple coroutines can be executed concurrently by one thread.

We use the asynchronous network connection of asyncio to get the home pages of sina, ES106 EN and 163:

import asyncio


@asyncio.coroutine
def wget(host):
 print('wget %s...' % host)
 connect = asyncio.open_connection(host, 80) #  Create Connection 
 reader, writer = yield from connect
 header = 'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % host
 writer.write(header.encode('utf-8'))
 yield from writer.drain()
 while True:
  line = yield from reader.readline()
  if line == b'\r\n':
   break
  print('%s header > %s' % (host, line.decode('utf-8').rstrip()))
 # Ignore the body, close the socket
 writer.close()


loop = asyncio.get_event_loop()
tasks = [wget(host) for host in ['www.sina.com.cn', 'www.sohu.com', 'www.163.com']]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

The results are as follows:

wget www.sohu.com...
wget www.sina.com.cn...
wget www.163.com...
(Wait for 1 period)
(Print out header for sohu)
www.sohu.com header > HTTP/1.1 200 OK
www.sohu.com header > Content-Type: text/html
...
(Print out header for sina)
www.sina.com.cn header > HTTP/1.1 200 OK
www.sina.com.cn header > Date: Wed, 20 May 2015 04:56:33 GMT
...
(Print out 163 header)
www.163.com header > HTTP/1.0 302 Moved Temporarily
www.163.com header > Server: Cdn Cache Server V2.0
...

You can see that three connections are concurrently completed by one thread through coroutine.

3. Summary

asyncio provides complete asynchronous IO support;

Asynchronous operations need to be done in coroutine through yield from;

Multiple coroutines can be encapsulated into a set of Tasks and executed concurrently.