Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: run celery without mingle, heartbeat, or gossip #68

Merged
merged 1 commit into from
Sep 17, 2024

Conversation

iloveagent57
Copy link
Member

@iloveagent57 iloveagent57 commented Sep 17, 2024

See https://www.cloudamqp.com/docs/celery.html for inspiration.

From above link.

Command-line arguments

Celery worker command-line arguments can decrease the message rates substantially. Place these options after the word 'worker' in your command line because the order of the celery options is strictly enforced in Celery 5.0. For example,

celery -A my_celery_app worker --without-heartbeat --without-gossip --without-mingle

Without these arguments Celery will send hundreds of messages per second with different diagnostic and redundant heartbeat messages. Unfortunately details about these settings have been removed from the current documentation, but the implementation has not changed. Read more about Celery worker functionality in the documentation.

We're hoping that disabling this issue resolves: https://2u-internal.atlassian.net/wiki/spaces/ENG/pages/1325236431/RCA+Sept+2024+edX+Celery+Queuing+issue

For future reference, we did not test these individually due to deploy time. If any one of these is needed for some reason with a future version of celery, it is possible that enabling one or two of these would be ok. That said, we also don't know if this resolves the issue.


Make sure that the following steps are done before merging:

  • Have a Site Reliability Engineer review the PR if you don't own all of the services impacted.
  • If you are adding any new default values that need to be overridden when this change goes live, update internal repos and add an entry to the top of the CHANGELOG.
  • Performed the appropriate testing.

@iloveagent57 iloveagent57 requested a review from robrap September 17, 2024 18:19
@iloveagent57 iloveagent57 force-pushed the aed/celery-command-no-mingle-etc branch from 8b12294 to 6bc6f82 Compare September 17, 2024 18:22
@iloveagent57 iloveagent57 force-pushed the aed/celery-command-no-mingle-etc branch from 6bc6f82 to 37df5bd Compare September 17, 2024 18:24
@robrap robrap merged commit aab2f94 into master Sep 17, 2024
4 checks passed
@robrap robrap deleted the aed/celery-command-no-mingle-etc branch September 17, 2024 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants