Emulating Stackless and greenlet
with each other
Péter Szabó, pts@google.com

EuroPython, Florence, 2011-06-20

What is a coroutine?

Most threads are waiting

AcceptConnections()
nbsocket.accept()
waiting for fd 3 becoming readable
HandleHttpProxyConnection()
ReadHttpRequest()
nbfile.readline()
nbsocket.recv()
waiting for fd 4 becoming readable
HandleHttpProxyConnection()
WriteHttpResponseHeader()
nbfile.write()
nbsocket.send()
waiting for fd 5 becoming writable
CleanCachePeriodically()
sleep()
sleeping until timestamp 123456789.01 is reached

Coroutine implementations in Python

... and the corresponding I/O libraries

Why emulate?

Why emulate coroutine implementations with each other?

Coroutine emulators in Python

greenlet emulation correctness

These numbers useless as metrics, thus they are not fair.

Stackless emulation correctness

These numbers useless as metrics, thus they are not fair.

Who needs 13371 concurrent connections?

Coroutines are also useful for just a dozen connections:

Memory leak in greenlet

import gc
import sys
from greenlet import greenlet
def F(other_ref, x):
  greenlet.getcurrent().parent.switch(42)
x = []
xr = sys.getrefcount(x)
g = [greenlet(F), greenlet(F)]
assert 42 == g[0].switch(g[1], x)
assert 42 == g[1].switch(g[0], x)  # Comment this line to fix.
del g[:]
gc.collect()
print sys.getrefcount(x) - xr  #: 4, greenlets still not deleted.

No memory leak in Stackless

import gc
import sys
import stackless
def F(other_ref, x):
  stackless.schedule_remove()
x = []
xr = sys.getrefcount(x)
g = [stackless.tasklet(F), stackless.tasklet(F)]
g[0](g[1], x)
g[1](g[0], x)
print sys.getrefcount(x) - xr  #: 2
g[0].run()
g[1].run()
del g[:]
print sys.getrefcount(x) - xr  #: 1, tasklets not deleted yet.
gc.collect()
print sys.getrefcount(x) - xr  #: 0, tasklets deleted.

Differences between greenlet, Stackless and coroutine

Emulation challenges

Pedantic:

Tricky:

?