筆記-python-coroutine

1.????? 協程

1.1.??? 協程的概念

協程，又稱微線程，纖程。英文名Coroutine。協程是一種用戶態的輕量級線程。

線程是系統級別的，它們是由操作系統調度；協程是程序級別的，由程序員根據需要自己調度。我們把一個線程中的一個個函數叫做子程序，那么子程序在執行過程中可以中斷去執行別的子程序；別的子程序也可以中斷回來繼續執行之前的子程序，這就是協程。也就是說同一線程下的一段代碼<1>執行著執行著就可以中斷，然后跳去執行另一段代碼，當再次回來執行代碼塊<1>的時候，接著從之前中斷的地方開始執行。

比較專業的理解是：

協程擁有自己的寄存器上下文和棧。協程調度切換時，將寄存器上下文和棧保存到其他地方，在切回來的時候，恢復先前保存的寄存器上下文和棧。因此：協程能保留上一次調用時的狀態(即所有局部狀態的一個特定組合)，每次過程重入時，就相當于進入上一次調用的狀態，換種說法：進入上一次離開時所處邏輯流的位置。

個人理解：

一個程序是一個任務組合，進程和線程是操作系統級的資源單位，這是任務細分的標準，程序依此來設計和調度。

有些時候需要在函數間更靈活的調度，又不想老是傳遞一堆變量，在編譯器級別設計了上下文管理器，可以在一個函數中中斷并切換到其它函數，然后又可以切回來繼續運行該函數。這種機制/體系就叫協程。

理論上這些調度可以使用函數，但會有大量數據重復，干脆在編譯器中設計好了，切走時保存好上下文，切回來時繼續執行下一個語句。

1.2.??? yield實現協程

代碼：

def consumer(name):

print(name)

r = ''

while True:

n = yield r

if not n:

return

print('consumer:%s %s ' %(n,name))

r = '200 OK'

def produce(c,d):

c.send(None)

d.send(None)

n = 0

while n < 5:

n = n + 1

print('producer:%s' %n)

r = c.send(n)

d.send(n)

print('producer: consumer return %s' %r)

c.close()

d.close()

c = consumer('B')

d = consumer('A')

produce(c,d)

2.????? greenlet

2.1.??? 開始的地方

官方文檔：https://greenlet.readthedocs.io/en/latest/

The “greenlet” package is a spin-off of?Stackless, a version of CPython that supports micro-threads called “tasklets”. T

Greenlets are provided as a C extension module for the regular unmodified interpreter.

使用案例1：

fromgreenlet importgreenlet

deftest1():

print(12)

gr2.switch()

print(34)

deftest2():

print(56)

gr1.switch()

print(78)

gr1 = greenlet(test1)

gr2 = greenlet(test2)

gr1.switch()

switch切換到gr1,gr2,gr1,然后就結束了，不會再切回到test2，print(78)不會執行。

整個greenlet可以理解為在執行代碼外包含一層環境，環境是嵌套的，可以切換。

官方解釋：

A “greenlet” is a small independent pseudo-thread. Think about it as a small stack of frames; the outermost (bottom) frame is the initial function you called, and the innermost frame is the one in which the greenlet is currently paused. You work with greenlets by creating a number of such stacks and jumping execution between them. Jumps are never implicit: a greenlet must choose to jump to another greenlet, which will cause the former to suspend and the latter to resume where it was suspended. Jumping between greenlets is called “switching”.

When you create a greenlet, it gets an initially empty stack; when you first switch to it, it starts to run a specified function, which may call other functions, switch out of the greenlet, etc. When eventually the outermost function finishes its execution, the greenlet’s stack becomes empty again and the greenlet is “dead”. Greenlets can also die of an uncaught exception.

既然是嵌套，一定會有父子，兄弟關系，并且很重要的一個問題是執行順序。

Remember, switches are not calls, but transfer of execution between parallel “stack containers”, and the “parent” defines which stack logically comes “below” the current one.。

2.2.??? 重要的屬性

dir(greenlet)列出屬性如下：

['GREENLET_USE_GC', 'GREENLET_USE_TRACING', 'GreenletExit', '_C_API', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__version__', 'error', 'getcurrent', 'gettrace', 'greenlet', 'settrace']

greenlet.greenlet?is the greenlet type, which supports the following operations:

greenlet(run=None,?parent=None)

Create a new greenlet object (without running it).?run?is the callable to invoke, and?parent?is the parent greenlet, which defaults to the current greenlet.

greenlet.getcurrent()

Returns the current greenlet (i.e. the one which called this function).

greenlet.GreenletExit

This special exception does not propagate to the parent greenlet; it can be used to kill a single greenlet.

The?greenlet?type can be subclassed, too. A greenlet runs by calling its?run?attribute, which is normally set when the greenlet is created; but for subclasses it also makes sense to define a?runmethod instead of giving a?run?argument to the constructor.

看一下greenlet.greenlet

>>> dir(greenlet.greenlet)

['GreenletExit', '__bool__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_stack_saved', 'dead', 'error', 'getcurrent', 'gettrace', 'gr_frame', 'parent', 'run', 'settrace', 'switch', 'throw']

run 當greenlet啟動時會調用到這個callable；啟動后屬性不復存在；

switch(*args, **kwargs) 切換

parent 父對象，可讀寫，但不允許存在循環；

gr_frame：the current top frame, or None;

dead: True if greenlet is dead(it finished its execution).

bool(g): True if?g?is active, False if it is dead or not yet started.

throw([typ, [val,[tb]]]):

Switches execution to the greenlet?g, but immediately raises the given exception in?g. If no argument is provided, the exception defaults to?greenlet.GreenletExit. The normal exception propagation rules apply, as described above. Note that calling this method is almost equivalent to the following:

def raiser():

raise typ, val, tb

g_raiser = greenlet(raiser, parent=g)

g_raiser.switch()

except that this trick does not work for the?greenlet.GreenletExit?exception, which would not propagate from?g_raiser?to?g.

2.3.??? switch

純切換沒什么好說的，帶參數則是另外一回事了。

Switches between greenlets occur when the method switch() of a greenlet is called, in which case execution jumps to the greenlet whose switch() is called, or when a greenlet dies, in which case execution jumps to the parent greenlet. During a switch, an object or an exception is “sent” to the target greenlet; this can be used as a convenient way to pass information between greenlets. For example:

def test1(x, y):

z = gr2.switch(x+y)

print(z)

def test2(u):

print(u)

gr1.switch(42)

gr1 = greenlet(test1)

gr2 = greenlet(test2)

gr1.switch("hello", " world")

切換傳值規則：

g.switch(*args,?**kwargs)

Switches execution to the greenlet?g, sending it the given arguments. As a special case, if?gdid not start yet, then it will start to run now.

Dying greenlet

If a greenlet’s?run()?finishes, its return value is the object sent to its parent. If?run()?terminates with an exception, the exception is propagated to its parent (unless it is a?greenlet.GreenletExitexception, in which case the exception object is caught and?returned?to the parent).

需要注意的是z = gr2.switch(x+y)，這個部分與yield的send類似，會接收傳遞的值。

Note that any attempt to switch to a dead greenlet actually goes to the dead greenlet’s parent, or its parent’s parent, and so on. (The final parent is the “main” greenlet, which is never dead.)

切換到dead greenlet的嘗試會切到目的greenlet的父級(或祖級)。

2.4.??? greenlets

協程是基于線程的，可以在線程中擁有一個main greenlet和一個子系greenlet樹。不能跨線程。

2.5.??? 垃圾回收

If all the references to a greenlet object go away (including the references from the parent attribute of other greenlets), then there is no way to ever switch back to this greenlet. In this case, a GreenletExit exception is generated into the greenlet. This is the only case where a greenlet receives the execution asynchronously. This gives?try:finally:?blocks a chance to clean up resources held by the greenlet.

一個greenlet所有的引用都停止時，會拋出異常。

代碼示例：

from greenlet import greenlet, GreenletExit

huge = []

def show_leak():

def test1():

gr2.switch()

def test2():

huge.extend([x* x for x in range(100)])

try:

gr1.switch()

finally:

print 'finish switch del huge'

del huge[:]

gr1 = greenlet(test1)

gr2 = greenlet(test2)

gr1.switch()

gr1 = gr2 = None

print 'length of huge is zero ? %s' % len(huge)

if __name__ == '__main__':

show_leak()

上述代碼的沒有正常結束(在第10行掛起)。第18行之后gr1,gr2的引用計數都變成0，那么會在第10行拋出GreenletExit異常，因此finally語句有機會執行。同時，在文章開始介紹Greenlet module的時候也提到了，GreenletExit這個異常并不會拋出到parent，所以main greenlet也不會出異常。

看上去貌似解決了問題，但這對程序員要求太高了，百密一疏。所以最好的辦法還是保證協程的正常結束。

Greenlets do not participate in garbage collection; cycles involving data that is present in a greenlet’s frames will not be detected. Storing references to other greenlets cyclically may lead to leaks.

還是注意協程的正常結束比較靠譜。

2.6.??? tracing support

greenlet的調試需要能夠理清它們之間的關系，好在不用每次自己去捋，greenlet模塊提供了這方面的支持。

greenlet.gettrace()

Returns a previously set tracing function, or None.

greenlet.settrace(callback)

Sets a new tracing function and returns a previous tracing function, or None. The callback is called on various events and is expected to have the following signature:

代碼示例：

def callback(event, args):

if event == 'switch':

origin, target = args

# Handle a switch from origin to target.

# Note that callback is running in the context of target

# greenlet and any exceptions will be passed as if

# target.throw() was used instead of a switch.

return

if event == 'throw':

origin, target = args

# Handle a throw from origin to target.

# Note that callback is running in the context of target

# greenlet and any exceptions will replace the original, as

# if target.throw() was used with the replacing exception.

return

代碼示例：

def callback_t(event, args):

print('{} from {} to {}'.format(event, id(args[0]), id(args[1])))

def test1(x, y):

z = gr2.switch(x+y)

print(z)

def test2(u):

print(u)

gr1.switch('wefweeger')

main = greenlet.getcurrent()

gr1 = greenlet(test1)

gr2 = greenlet(test2)

print('main is {}, gr1 is {}, gr2 is {}.'.format(id(main), id(gr1), id(gr2)))

oldtrace = greenlet.settrace(callback_t)

gr1.switch("hell", " world")

2.7.??? 總結

greenlet類似于棧

greenlet創生之后，一定要結束，不能switch出去就不回來了，否則容易造成內存泄露；

greenlet不能跨線程；

不能存在循環引用，這個是官方文檔明確說明；

3.????? gevent

doc address: http://www.gevent.org/contents.html

gevent is a?coroutine?-based?Python?networking library that uses?greenlet?to provide a high-level synchronous API on top of the?libev?or?libuv?event loop.

比較長，另開一章。

總結

以上是生活随笔為你收集整理的python coroutine_笔记-python-coroutine的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。