Track weight proof tasks

[almogdepaz]

Purpose:

track weight proof segment creation tasks until closed

Current Behavior:

we cancel tasks and stop tracking them

New Behavior:

keep a list of all tasks and remove when they are done

Testing Notes:

[arvidn]

I believe you also need to await any remaining tasks on shutdown

[arvidn]

@altendky Is there any important distinction between checking done() and drop the reference vs. await ing the task (even when it's done)?

[altendky]

$ cat z.py
import asyncio

async def f():
    await asyncio.sleep(0.1)
    return 37

async def main():
    t = asyncio.create_task(f())
    print(await t)
    print(await t)
    c = f()
    print(await c)
    print(await c)

asyncio.run(main())

$ python z.py
37
37
37
Traceback (most recent call last):
  File "/home/altendky/z.py", line 15, in <module>
    asyncio.run(main())
  File "/home/altendky/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/altendky/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/altendky/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/altendky/z.py", line 13, in main
    print(await c)
          ^^^^^^^
RuntimeError: cannot reuse already awaited coroutine

it looks like double awaiting a task is fine and you even get the result again. i added the separate case of double awaiting a coroutine object since i remembered there being an issue here and figured i'd point out the difference between the two cases.

[arvidn]

I think of await as join()ing a thread. It seems right to always "join"

[altendky]

without having simple hard backing of the importance, yes, i would generally agree that all coroutines and tasks ought to be awaited. have to consider errors coming out of tasks and coroutines though, unlike threads.

[altendky]

A general question. Is it ok to have multiple segment tasks concurrently? Yes, we request cancellation but we don't wait for that to be completed before launching the new task. Hence the relevance of the list.

I do think that we need a systemic resolution to the general complexity of properly handling tasks. I think the solution is anyio. There are some steps that are needed before we do that. In the mean time we're basically just patching holes, adding stop gaps, etc.

[altendky]

A general question. Is it ok to have multiple segment tasks concurrently? Yes, we request cancellation but we don't wait for that to be completed before launching the new task. Hence the relevance of the list.

[github-actions]

File	Coverage	Missing Lines
chia/full_node/full_node.py	91.7%	lines 611

Total	Missing	Coverage
12 lines	Unknown	91%

n/a