Introduction to concurrent programming using the greenlet package in Python

1     motivation

The greenlet package is a byproduct of Stackless, which refers to tasklets as "tasklet." tasklet runs in pseudo-concurrency, using channel for synchronous data exchange.

One "greenlet" is a more primitive concept of tasklets, but there is no scheduling, or coroutine. This is useful when you need to control your code. You can build your own micro thread scheduler; You can also use "greenlet" for advanced control flow. For example, you can recreate the constructor; Unlike Python's constructor, our constructor can nest calls to functions, and the nested function can yield with 1 value. (also, you don't need an "yield" keyword, see the example).

Greenlet is given to the unmodified interpreter as an C extension module.

1.1 examples of    

Assume that the system is controlled by a console program, with user input commands. Suppose the input is 1 character per input. The system looks like this:

def process_commands(*args):
  while True:
    line=''
    while not line.endswith('\n'):
      line+=read_next_char()
    if line=='quit\n':
      print "are you sure?"
      if read_next_char()!="y":
        continue  # Ignore instructions 
    process_commands(line)

Now suppose you want to port your program to GUI, and most GUI are event-driven. They call the callback function every time the user enters. In this case, it is difficult to implement the read_next_char() function. We have two incompatible functions:

def event_keydown(key):
      ??

def read_next_char():
      & # 63; The & # 63; You need to wait for a call to event_keydown()

You might be thinking about using threads. While Greenlet is another solution, there is no lock and close problem. You start the process_commands() function, split it into greenlet, and then interact with the keystroke event as follows:

def event_keydown(key):
  g_processor.switch(key)

def read_next_char():
  g_self=greenlet.getcurrent()
  next_char=g_self.parent.switch()  # Jump on 1 layer (main) the greenlet And wait for the 1 A key 
  return next_char

g_processor=greenlet(process_commands)
g_processor.switch(*args)
gui.mainloop()

The execution flow of this example is: read_next_char() is called, which is part 1 of g_processor, and it will switch (switch) to its parent greenlet, assuming that it continues to execute in the top-level main loop (GUI main loop). When GUI calls event_keydown(), it switches to g_processor, which means that execution jumps back to where it was suspended, where the switching instruction in the read_next_char() function was. The key parameter of event_keydown() is then passed to the switch of read_next_char() and returned.

Note that read_next_char() will be suspended and assume that its call stack will be well protected when restored, so it will return where it was called. This allows the program logic to maintain a graceful sequential flow. We don't have to rewrite process_commands() to use a state machine.

2     use

2.1 introduction to    

One "greenlet" is a small, independent tasklet. Think of it as a stack frame, where the bottom of the stack is the initial call and the top of the stack is the current greenlet pause. You use greenlet to create stacks like 1 heap and jump between them. Jumps are not absolute: one greenlet must choose to jump to another greenlet, which makes the first one hang and the second one resume. A jump between two greenlet is called a switch (switch).

When you create an greenlet, it gets an empty stack initialized. When you switch to it the first time, it will launch the specified function and then switch out of greenlet. When the bottom function ends, greenlet's stack is programmed to be empty again, and greenlet dies. greenlet will also die due to an uncaught exception.

Such as:

from py.magic import greenlet

def test1():
  print 12
  gr2.switch()
  print 34

def test2():
  print 56
  gr1.switch()
  print 78

gr1=greenlet(test1)
gr2=greenlet(test2)
gr1.switch()

The last line jumps to test1(), which prints 12, then jumps to test2(), prints 56, then jumps back to test1(), prints 34, then test1() ends, gr1 dies. The execution then returns to the original gr1.switch () call. Note that 78 will not be printed.

2.2     parent greenlet

Now let's see where the execution point goes when an greenlet dies. Each greenlet has one parent greenlet. The parent greenlet is created at the beginning of each greenlet (though it can be changed at any time). The parent greenlet executes when greenlet dies. Thus, greenlet is organized into a tree, and the top-level code does not run in the user-created greenlet, but is called master greenlet, or root.

In the example above, gr1 and gr2 both have master greenlet as the parent greenlet. Any 1 that dies, the execution point goes back to the main function.

Uncaught exceptions will affect the parent greenlet. If test2() above contains 1 typo (typo), it will generate 1 NameError and kill gr2, then the execution point will return to the main function. traceback will display test2() instead of test1(). Remember, a switch is not a call, but execution points can be exchanged in parallel between parallel stack containers, and the parent greenlet defines where the stack originally came from.

2.3     instance

py.magic.greenlet is a type of greenlet and supports the following operations:

greenlet(run=None,parent=None)

      creates an greenlet object without executing it. run is a callback, parent is the parent greenlet, and the default is the current greenlet.

greenlet.getcurrent()

      returns the current greenlet, that is, who is calling this function.

greenlet.GreenletExit

      this particular exception does not affect the parent greenlet, it is used to kill an greenlet.

The greenlet type can be inherited. An greenlet is executed by calling its run property, which is the one specified at creation time. For subclasses, you can define one run() method without strictly following the run parameter given in the constructor.

2.4     switching

Switching between greenlet occurs when greenlet's switch() method is called, which jumps the execution point to where greenlet's switch() is called. Or when greenlet dies, jump to the parent greenlet. When switching, an object or exception is sent to the target greenlet. This can be used as a convenient way to transfer information between the two greenlet. Such as:

def test1(x,y):
  z=gr2.switch(x+y)
  print z

def test2(u):
  print u
  gr1.switch(42)

gr1=greenlet(test1)
gr2=greenlet(test2)
gr1.switch("hello"," world")

This will print "hello world" and 42, in the same order as the previous example. Note that the parameters for test1() and test2() are not specified when greenlet is created, but are passed the first time it is switched here.

Here's the exact invocation:

g.switch(obj=None or *args)

Switch to the execution point greenlet g and send the given object obj. In special cases, if g is not already started, it will be started. In this case, a parameter is passed and g.run (*args) is called.

The dying greenlet

If an       greenlet run() ends, it returns the value to the parent greenlet. If run() terminates abnormally, the exception will affect the parent greenlet(unless it is an greenlet.GreenletExit exception, in which case the exception will be caught and returned to the parent greenlet).

In addition to the above, the target greenlet receives the sent object as the return value of switch(). Although switch() does not return immediately, it will still return at some point in the future, when the other greenlet switches back. When this happens, the execution point reverts to switch(), which returns the object the caller just sent. This means that x= g.switch (y) will send an object y to g, and then wait for an object that you don't know who sent it to, and return it to x here.

Note that any attempt to switch to dead greenlet will switch to dead greenlet's father, greenlet's father, and so on. The ultimate father is main greenlet and will never die.

2.5 methods and properties of     greenlet

g.switch(obj=None or *args)

      switches the execution point to greenlet g, ibid.

g.run

      calls the executable g and starts. This property no longer exists after g is started.

g.parent

      greenlet father. This is writable, but does not allow creating a parent relationship for the loop.

g.gr_frame

      current top-level frames, or None.

g.dead

     

bool(g)

      returns True if g is active, and False before it has been started or finished.

g.throw([typ,[val,[tb]]])

      switches the execution point to greenlet g, but immediately throws the specified exception to g. If no arguments are provided, the exception defaults to greenlet.GreenletExit. According to the exception sweep rule, as described above. Note that calling this method is equivalent to:

  def raiser():
    raise typ,val,tb

  g_raiser=greenlet(raiser,parent=g)
  g_raiser.switch()

2.6     Greenlet and Python threads

greenlet can be used with Python thread 1 up; In this case, each thread contains a separate main greenlet and has its own greenlet tree. Different threads may not switch greenlet to each other.

2.7     activity greenlet garbage collection

If there is no longer a reference to the greenlet object (including the parent of other greenlet), there is no way to switch back to greenlet. In this case, 1 GreenletExit exception is generated to greenlet. This is the only case where greenlet receives an asynchronous exception. Should give 1 try.. finally is used to clean up resources within greenlet. This feature also allows for an infinite loop programming style in greenlet. This allows the loop to break automatically when the last reference disappears.

If you don't want greenlet to die or put the reference somewhere else, just catch and ignore the GreenletExit exception.

greenlet does not participate in garbage collection; The circular reference data for frame greenlet is detected. Passing a reference to another loop, greenlet, can cause a memory leak.