Change Celery broker to rabbitmq #44
Conversation
|
@gherceg @AmitPhulera |
| log.federation.level = warning | ||
| log.mirroring.level = warning | ||
| consumer_timeout = 86400001 | ||
| heartbeat = 600 |
There was a problem hiding this comment.
Just a potential gotcha, celery also sets a broker_timeout, and the lower of that value and this heartbeat value is what is actually used.
environments/staging/vars.yml
Outdated
| @@ -12,6 +12,8 @@ superset_db_name: superset_meta | |||
| oauth2_db_name: superset_oauth2 | |||
| cchq_db_name: superset_cchq_data | |||
|
|
|||
| rabbitmq_version: 3.9.13-1 | |||
There was a problem hiding this comment.
Any specific reason for not using the latest version of rabbitmq? https://www.rabbitmq.com/release-information
There was a problem hiding this comment.
I took the one that was showing up in the available versions for the server I am installing this on, 22.04.4 LTS.
I see HQ is using 3.10.13-1, So I will be okay to jump to that at least, if its available.
There was a problem hiding this comment.
If this is a fresh install, I would suggest go with the latest supported version, which I believe is 4. This would be helpful in longer run where you will save some time in atleast one of the upgrades.
There was a problem hiding this comment.
the environment is existing but installing rabbit is new, I can try a more recent version on staging first 👍
There was a problem hiding this comment.
It appears the version being used by HQ is the latest one available via apt. Anything above that version gives a long error message complaining about many depenedency though mainly probably with erlang version.
Looks like we need to change how we install erlang.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
rabbitmq-server : Depends: erlang-base (>= 1:24.3.2) but 1:24.2.1+dfsg-1ubuntu0.1 is to be installed or
erlang-base-hipe (>= 1:24.3.2) but it is not installable or
esl-erlang (>= 1:24.3.2) but it is not installable
Depends: erlang-crypto (>= 1:24.3.2) but 1:24.2.1+dfsg-1ubuntu0.1 is to be installed or
esl-erlang (>= 1:24.3.2) but it is not installable
Depends: erlang-eldap (>= 1:24.3.2) but 1:24.2.1+dfsg-1ubuntu0.1 is to be installed or
esl-erlang (>= 1:24.3.2) but it is not installable
Depends: erlang-inets (>= 1:24.3.2) but 1:24.2.1+dfsg-1ubuntu0.1 is to be installed or
...
So, using the same version as on HQ which I can update to without breaking anything else.
| owner: rabbitmq | ||
| mode: 0644 | ||
| loop: | ||
| - {"src": "rabbitmq/rabbitmq.conf.j2", "dest": "rabbitmq.conf"} |
There was a problem hiding this comment.
Since there is only one element in the loop, would it make sense to have static values for source and dest above?
There was a problem hiding this comment.
Ya, I held on to it from the cluster implementation I copied over and thought of retaining it for future.
mkangia
force-pushed
the
mk/4028-change-celery-backend
branch
from
7aa447f to
dce784e
Compare
https://dimagi.atlassian.net/browse/SC-4028
This changes celery broker from redis to rabbitmq by default.
No changes to result backend being redis, since that is working okay.
Also, rabbitmq as broker and celery as result backend, is a common setup for celery.
The setup has been picked from commcare cloud's rabbitmq setup keeping just the relevant steps for CommCare Analytics
Staging is now using rabbitmq as broker & successfully tested by sending a couple of data source repeaters.
Roll out to production to be scheduled for since the switch needs an empty backlog on celery.