Skip to content

0.1.32

Compare
Choose a tag to compare
@tgbugs tgbugs released this 14 Jan 00:59
utils Async critical fix for variant behavior on pypy3, vbump

to make a very long debug story much shorter: it turns out that pypy3
and cpython have remarkably and dangerously different behavior for
closures in comprehensions

fixing the bad closure behavior also seems to have resolved the
wild and crazy hitching and slowdowns seen toward the end of long
async runs, probably because i was also changing

these have obvious behavior that is consistent for both
pypy3 and cpython

help = (lambda : OH_NO for OH_NO in range(10))
me = list(help)
sigh = [m() for m in me]

help = (lambda : OH_NO for OH_NO in range(10))
sigh = [h() for h in help]

however consider

from time import sleep
import asyncio
from concurrent.futures import ThreadPoolExecutor
def OH_NO_NO_NO():
    ex = ThreadPoolExecutor()
    help = (lambda : (OH_NO, sleep((OH_NO + 1) / 10)) for OH_NO in range(10))
    #help = (lambda : OH_NO for OH_NO in range(10))
    async def ohno(future_):
        loop = asyncio.get_event_loop()
        futures = []
        for f in help:
            futures.append(loop.run_in_executor(ex, f))
        out = []
        for f in futures:
            out.append(await f)
            sleep(0.1)
        future_.set_result(out)

    future = asyncio.Future()
    loop = asyncio.get_event_loop()
    loop.run_until_complete(ohno(future))
    return future.result()

cpython also fails in this case, but if you look at pypy3 the behavior
is significantly more variable, in the context of Async there is some
additional factor which has somehow ensured that cpython chunk didn't
get rebound ... or something like that ... I haven't completed the
investigation with a full explaination, but this is at least part of
the story