Mettre à jour Celery sur la dernière version

[maudetes]

Plusieurs fois, suite à un crash d'un worker celery, le worker a cessé de dépiler des tâches lors du restart.
Exemple de logs avec un WorkerLostError qui n'a ensuite pas dépiler de tâches au restart.

[2024-12-31 01:29:20,178: INFO/ForkPoolWorker-22630] Task udata.search.reindex[9daade56-bf99-4930-940d-ad8cf28c2405] succeeded in 0.21213988214731216s: None
[2024-12-31 01:29:20,181: INFO/MainProcess] Task udata.search.reindex[157074a4-d3a1-4c85-9c9d-768b92a35e66] received
[2024-12-31 01:29:20,332: INFO/ForkPoolWorker-22630] Task udata.search.reindex[20f50f2e-ce98-4f07-881f-4ce65f807553] succeeded in 0.1516287364065647s: None
[2024-12-31 01:29:20,335: INFO/MainProcess] Task udata.search.reindex[680bd3a7-427d-45ad-b280-c4c4d87fe269] received
[2024-12-31 01:29:20,473: INFO/ForkPoolWorker-22630] Task udata.search.reindex[3a70140b-3a29-4670-b77f-1cc119019dfa] succeeded in 0.1391371414065361s: None
[2024-12-31 01:29:20,476: INFO/MainProcess] Task udata.search.reindex[d7a6205f-17db-4283-b754-c2be0c7043d1] received
[2024-12-31 01:29:20,671: INFO/ForkPoolWorker-22630] Task udata.search.reindex[55382253-80f0-4b2d-a336-3bf6af704ca5] succeeded in 0.19618766754865646s: None
[2024-12-31 01:29:20,675: INFO/MainProcess] Task udata.search.reindex[ab6cd268-6d86-4cc8-8681-3d634c2a88eb] received
[2024-12-31 01:29:20,803: INFO/ForkPoolWorker-22630] Task udata.search.reindex[64e95dfb-356f-4567-960b-577122841dcf] succeeded in 0.12962151318788528s: None
[2024-12-31 01:29:20,983: INFO/ForkPoolWorker-22630] Task udata.search.reindex[f6a96f50-2008-4f6b-86a9-e13768d5e324] succeeded in 0.17810652032494545s: None
[2024-12-31 01:29:21,031: INFO/MainProcess] Task udata.search.reindex[2ab2cf55-a462-436d-8714-85189ce90e0c] received
[2024-12-31 01:29:21,033: INFO/MainProcess] Task udata.search.reindex[1fd1a7b8-a839-448b-9311-1bc5a3508104] received
[2024-12-31 01:29:21,342: INFO/ForkPoolWorker-22630] Task udata.search.reindex[01a34f18-b4d5-4145-b0bf-f9e9f968f123] succeeded in 0.3101743385195732s: None
[2024-12-31 01:29:21,346: INFO/MainProcess] Task udata.search.reindex[0cca8309-2853-4016-8de2-538a31ce446f] received
[2024-12-31 01:29:21,418: ERROR/MainProcess] Process 'ForkPoolWorker-22630' pid:2493824 exited with 'signal 11 (SIGSEGV)'
[2024-12-31 01:29:21,438: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: signal 11 (SIGSEGV) Job: 2233219.')
Traceback (most recent call last):
  File "/srv/demo/lib/python3.11/site-packages/billiard/pool.py", line 1264, in mark_as_worker_lost
    raise WorkerLostError(
billiard.einfo.ExceptionWithTraceback: 
"""
Traceback (most recent call last):
  File "/srv/demo/lib/python3.11/site-packages/billiard/pool.py", line 1264, in mark_as_worker_lost
    raise WorkerLostError(
billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 11 (SIGSEGV) Job: 2233219.
"""
[2024-12-31 01:29:21,550: WARNING/MainProcess] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: DECRYPTION_FAILED_OR_BAD_R
ECORD_MAC] decryption failed or bad record mac (_ssl.c:2546)'))': /api/12/envelope/
[2024-12-31 01:29:21,585: INFO/MainProcess] Task udata.search.reindex[ad5a5058-23d4-4c14-b343-f621b302d121] received
[2024-12-31 07:29:21,583: WARNING/MainProcess] Soft time limit (21600s) exceeded for udata.search.reindex[47ee90b4-7fbf-4e87-b693-5d7cd737dbe0]
[2025-01-07 13:10:36,463: INFO/MainProcess] missed heartbeat from low@dev-03
[2025-01-08 07:20:42,904: INFO/MainProcess] missed heartbeat from low@dev-03
[2025-01-08 08:45:58,367: INFO/MainProcess] missed heartbeat from low@dev-03
[2025-01-08 09:30:50,845: INFO/MainProcess] missed heartbeat from low@dev-03
[2025-01-08 11:20:27,934: INFO/MainProcess] missed heartbeat from low@dev-03

Un restart manuelle de celery permet bien de relancer le worker de manière fonctionnelle par la suite.

Cela semble pouvoir être lié à la version actuelle de Celery + Kombu.