asyncio, telegrambot SSL and event loop closed issues

[dncastilho]

I have apscheduler running function "frequent_jobs" every 2 minutes, to do a few sync methods and use the async method below to send messages through telegram bot. Every now and then, I face an error (also below). Any idea why this might be happening? OS is ubuntu, python 3.10.7, telegrambot latest (4.10.0).

protocol: <asyncio.sslproto.SSLProtocol object at 0x7fe965351f00>
transport: <_SelectorSocketTransport closing fd=29>
Traceback (most recent call last):

  File "/usr/lib/python3.10/asyncio/selector_events.py", line 916, in write
    n = self._sock.send(data)
        │    │     │    └ b'\x17\x03\x03\x00\x13]E\x8c\xef7\xb3z\x9a\x17\x82 \na\xecsI\xd5SS'
        │    │     └ <method 'send' of '_socket.socket' objects>
        │    └ <socket.socket fd=-1, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6>
        └ <_SelectorSocketTransport closing fd=29>

OSError: [Errno 9] Bad file descriptor


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
    │    └ <function Thread._bootstrap_inner at 0x7fe97860d630>
    └ <Thread(ThreadPoolExecutor-0_0, started 140640415729344)>
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
    │    └ <function Thread.run at 0x7fe97860d360>
    └ <Thread(ThreadPoolExecutor-0_0, started 140640415729344)>
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)

...

 File "/usr/lib/python3.10/asyncio/sslproto.py", line 321, in __del__
    self.close()
    │    └ <function _SSLProtocolTransport.close at 0x7fe9785ff1c0>
    └ <asyncio.sslproto._SSLProtocolTransport object at 0x7fe96527af80>
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 316, in close
    self._ssl_protocol._start_shutdown()
    │    │             └ <function SSLProtocol._start_shutdown at 0x7fe9785ffe20>
    │    └ <asyncio.sslproto.SSLProtocol object at 0x7fe965351f00>
    └ <asyncio.sslproto._SSLProtocolTransport object at 0x7fe96527af80>
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 599, in _start_shutdown
    self._write_appdata(b'')
    │    └ <function SSLProtocol._write_appdata at 0x7fe9785ffeb0>
    └ <asyncio.sslproto.SSLProtocol object at 0x7fe965351f00>
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 604, in _write_appdata
    self._process_write_backlog()
    │    └ <function SSLProtocol._process_write_backlog at 0x7fe978338160>
    └ <asyncio.sslproto.SSLProtocol object at 0x7fe965351f00>
> File "/usr/lib/python3.10/asyncio/sslproto.py", line 690, in _process_write_backlog
    self._transport.write(chunk)
    │    │          │     └ b'\x17\x03\x03\x00\x13]E\x8c\xef7\xb3z\x9a\x17\x82 \na\xecsI\xd5SS'
    │    │          └ <function _SelectorSocketTransport.write at 0x7fe9783f43a0>
    │    └ <_SelectorSocketTransport closing fd=29>
    └ <asyncio.sslproto.SSLProtocol object at 0x7fe965351f00>
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 922, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
    │    └ <function _SelectorTransport._fatal_error at 0x7fe9783cfb50>
    └ <_SelectorSocketTransport closing fd=29>
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 717, in _fatal_error
    self._force_close(exc)
    │    │            └ OSError(9, 'Bad file descriptor')
    │    └ <function _SelectorTransport._force_close at 0x7fe9783cfbe0>
    └ <_SelectorSocketTransport closing fd=29>
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 729, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
    │    │               │    │                      └ OSError(9, 'Bad file descriptor')
    │    │               │    └ <function _SelectorSocketTransport._call_connection_lost at 0x7fe9783f45e0>
    │    │               └ <_SelectorSocketTransport closing fd=29>
    │    └ <member '_loop' of '_FlowControlMixin' objects>
    └ <_SelectorSocketTransport closing fd=29>
  File "/usr/lib/python3.10/asyncio/base_events.py", line 750, in call_soon
    self._check_closed()
    │    └ <function BaseEventLoop._check_closed at 0x7fe97839e290>
    └ <_UnixSelectorEventLoop running=False closed=True debug=False>
  File "/usr/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')

Code

async def send_one_message(semaphore: asyncio.Semaphore, chat_id, message, session):
    async with semaphore:
        try:
            await telegram_bot.send_driver_notification(chat_id, message)
            await asyncio.sleep(5) #throttle
        

async def send_notifications():
    semaphore = asyncio.Semaphore(20)
    logger.debug(f"start send_notifications")
    message_sending = []
    try:
        with Session(engine) as session:
            for message in session.exec(select(NotificationMessage)).all():
                message_sending.append(asyncio.create_task(send_one_message(semaphore, message.chat_id, message.text)))
            if len(message_sending) > 0:
                await asyncio.wait(message_sending)
    finally:
        telegram_bot.bot.close_session() #close session after each batch
    logger.debug(f"end send_notifications")

def frequent_jobs(test_mode: bool = False):
    some_other_stuff()
    asyncio.run(send_notifications())
    more_stuff()

[coder2020official]

Try getting/creating loop and using it instead of asyncio.run

[dncastilho]

I have reproduced the same error message with the following simple code

from telebot.async_telebot import AsyncTeleBot
bot = AsyncTeleBot(BOT_TOKEN, parse_mode="html")
import asyncio

if __name__ == "__main__":
    asyncio.run(bot.send_message(CHAT_ID, 'test'))
    asyncio.run(bot.close_session())

Error is:

Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f79330c5990>
transport: <_SelectorSocketTransport closing fd=6>
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 916, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 690, in _process_write_backlog
    self._transport.write(chunk)
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 922, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 717, in _fatal_error
    self._force_close(exc)
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 729, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 750, in call_soon
    self._check_closed()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f79330c5690>
transport: <_SelectorSocketTransport closing fd=7>
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 916, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 690, in _process_write_backlog
    self._transport.write(chunk)
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 922, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 717, in _fatal_error
    self._force_close(exc)
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 729, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 750, in call_soon
    self._check_closed()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

[coder2020official]

Are u using windows?

[dncastilho]

Are u using windows?

no, the above test was done on ubuntu. more specifically

5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Ubuntu on WSL. But the same happens in a ubuntu cloud server I have, which is ubuntu 22.10.

[coder2020official]

Hmm that's interesting. Try this, this should fix the issue on windows, maybe there'll be an effect on Ubuntu too:
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())

[dncastilho]

in windows i am not sure the issue happens, i havent tested. but in ubuntu this doesn't work (it can't even find the policy)