Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7601] Install and configure Celery #2523

Merged
merged 5 commits into from
Sep 25, 2023
Merged

Conversation

hklarner
Copy link
Contributor

@hklarner hklarner commented Aug 22, 2023

No description provided.

@hklarner hklarner requested a review from m4ra August 22, 2023 15:44
@github-actions
Copy link

github-actions bot commented Aug 22, 2023

Coverage report

Total coverage

Status Category Percentage Covered / Total
🔴 Statements 11.88% 84/707
🔴 Branches 13.41% 48/358
🔴 Functions 9.73% 25/257
🔴 Lines 23.18% 529/2282

Status of coverage: 🟢 - ok, 🟡 - slightly more than threshold, 🔴 - under the threshold

Report generated by 🧪jest coverage report action from 30a6e10

@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from cf9061e to 5b8ee85 Compare August 24, 2023 09:15
adhocracy-plus/config/settings/base.py Outdated Show resolved Hide resolved
adhocracy-plus/config/celery.py Outdated Show resolved Hide resolved
docs/celery.md Show resolved Hide resolved
adhocracy-plus/config/celery.py Outdated Show resolved Hide resolved
docs/celery.md Outdated Show resolved Hide resolved
docs/celery.md Outdated Show resolved Hide resolved
requirements/base.txt Show resolved Hide resolved
apps/__init__.py Outdated Show resolved Hide resolved
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from 5b8ee85 to 3714b34 Compare September 5, 2023 14:49
@hklarner hklarner requested review from m4ra and goapunk September 5, 2023 14:49
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch 2 times, most recently from 4cd21c8 to ab3d95a Compare September 5, 2023 15:20
Copy link
Contributor

@m4ra m4ra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good!
just few tiny corrections

adhocracy-plus/config/settings/base.py Outdated Show resolved Hide resolved
requirements/base.txt Show resolved Hide resolved
adhocracy-plus/config/settings/base.py Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@hklarner hklarner changed the title [7601] Install and configure Celery WIP [7601] Install and configure Celery Sep 7, 2023
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from 30a6e10 to 3a7e328 Compare September 7, 2023 15:31
@hklarner hklarner added the Dev: A4 depending PR or issue dependent on A4 label Sep 7, 2023
@hklarner hklarner requested review from m4ra and goapunk September 7, 2023 15:32
@hklarner hklarner changed the title WIP [7601] Install and configure Celery [7601] Install and configure Celery Sep 11, 2023
README.md Show resolved Hide resolved
docs/celery.md Show resolved Hide resolved
apps/projects/tasks.py Show resolved Hide resolved
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch 4 times, most recently from 9c98260 to 341bd17 Compare September 13, 2023 07:30
Copy link
Contributor

@m4ra m4ra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

background task need the migration to zero as we discussed, then we can remove them from settings + requirements

@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from a34a8a4 to 80f4d5b Compare September 14, 2023 08:23
Makefile Show resolved Hide resolved
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from 80f4d5b to a5805f8 Compare September 14, 2023 14:09
@hklarner hklarner requested review from goapunk and m4ra September 14, 2023 14:18
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from b735cc7 to 2907d0e Compare September 18, 2023 17:59
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch 2 times, most recently from abf24cb to 47ca74b Compare September 20, 2023 12:17
@hklarner
Copy link
Contributor Author

I have added a new celery configuration parameter:

CELERY_RESULT_EXTENDED = True

that instructs celery to save more information in the results backend. So what used to be:

$ redis-cli GET celery-task-meta-8b3be356-9e50-4c41-8b62-ec766504bf04 | jq
{
  "status": "SUCCESS",
  "result": null,
  "traceback": null,
  "children": [],
  "date_done": "2023-09-19T12:33:11.118701",
  "task_id": "8b3be356-9e50-4c41-8b62-ec766504bf04"
}

is now

$ redis-cli GET celery-task-meta-467e1aff-a7c6-4fb8-aa47-093aa86150d8 | jq
{
  "status": "SUCCESS",
  "result": null,
  "traceback": null,
  "children": [],
  "date_done": "2023-09-20T07:53:10.285548",
  "name": "adhocracy4.emails.tasks.send_async",
  "args": [],
  "kwargs": {
    "email_module_name": "apps.notifications.emails",
    "email_class_name": "NotifyCreatorOnModeratorFeedback",
    "app_label": "a4_candy_ideas",
    "model_name": "idea",
    "object_pk": 62,
    "args": [],
    "kwargs": {}
  },
  "worker": "celery@tuxedo86",
  "retries": 0,
  "queue": "celery",
  "task_id": "467e1aff-a7c6-4fb8-aa47-093aa86150d8"
}

@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from 47ca74b to e5dc7a8 Compare September 20, 2023 14:35
@hklarner
Copy link
Contributor Author

As discussed with goapunk, I have added redis == 5.0.0 back to requirements/dev.txt to avoid this attribute error when developing locally:

AttributeError: 'NoneType' object has no attribute 'Redis'

apps/projects/tasks.py Outdated Show resolved Hide resolved
apps/projects/tasks.py Outdated Show resolved Hide resolved
apps/notifications/emails.py Show resolved Hide resolved
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from e5dc7a8 to 07add1a Compare September 21, 2023 11:14
@hklarner hklarner requested review from m4ra and goapunk September 21, 2023 11:16
requirements/prod.txt Outdated Show resolved Hide resolved
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from 07add1a to 6980b13 Compare September 21, 2023 12:00
@hklarner hklarner requested a review from goapunk September 21, 2023 12:01
docs/celery.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from 6980b13 to eb7ab6d Compare September 21, 2023 14:23
@hklarner hklarner requested a review from m4ra September 21, 2023 14:23
Copy link
Contributor

@m4ra m4ra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ready to merge once we merge a4 PR and update the requirements here

@hklarner hklarner force-pushed the hk-2023-08-install-and-configure-celery branch from eb7ab6d to 10c2617 Compare September 25, 2023 07:38
@hklarner
Copy link
Contributor Author

Looks ready to merge once we merge a4 PR and update the requirements here

I have replaced the last version bump commit with one that imports the latest a4, namely db03f3.

@hklarner hklarner requested a review from m4ra September 25, 2023 07:46
Copy link
Contributor

@goapunk goapunk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work! Will leave for @m4ra to give final review and merge

@m4ra m4ra merged commit 37f21d1 into main Sep 25, 2023
2 checks passed
@m4ra m4ra deleted the hk-2023-08-install-and-configure-celery branch September 25, 2023 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dev: A4 depending PR or issue dependent on A4
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants