Iteration over results from oapi.wrap_for_multiproc raising KeyError: '__getstate__'

[chimezie]

I originally got this error in my code while iterating over the results gathered asynchronously from a call to oapi.wrap_for_multiproc(prompt_to_chat('.. prompt ..'), **model_params). I was able to successfully reproduce it by running demo/chat_web_selects.py from the main branch:

.concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/multiprocessing/queues.py", line 244, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
KeyError: 'getstate'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
[..]
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/chimezie/Projects/Oori/OgbujiPT/demo/chat_web_selects.py", line 151, in async_main
retval = next(iter(done)).result()
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/site-packages/ogbujipt/llm_wrapper.py", line 359, in schedule_callable
response = await loop.run_in_executor(executor, prepped_callable, *args)
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/multiprocessing/queues.py", line 244, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/chimezie/miniconda3/envs/OgbujiPT-py-3.10/lib/python3.10/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
KeyError: 'getstate'

This is how it was run in my command line

python demo/chat_web_selects.py --apibase http://myserver:8000 --question "How old are the Igbo people?" "http://en.wikipedia.org/wiki/Igbo_people|ahiajoku.igbonet.com/2000/|http://en.wikivoyage.org/wiki/Igbo_phrasebook"

I'm running Python 3.10.12 in a Conda environment with OgbujiPT built from the main branch

[uogbuji]

Finally got a moment to look into this. I've never used conda, but I can't reproduce on 3.11 using regular virtual env. Strangely, I don't get any output for your "How old are the Igbo people?" question using openhermes-2.5-mistral-7b-16k.Q5 but when I change the question to "Where do the Igbo people live?" I get a good answer.

Just reading the error you're getting it seems to think there is an unpicklable object when crossing the fork boundary, but I can't think what that might be, and usually the error is clearer in terms of what can't be pickled.