Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

removed the wait on case upload record #34741

Merged
merged 1 commit into from
Jul 12, 2024
Merged

Conversation

mjriley
Copy link
Contributor

@mjriley mjriley commented Jun 7, 2024

Technical Summary

The case uploader needs to create a case upload record that contains the task ID. Currently, that is done by first creating the task so that we have a task id, then generated the case upload record, and then finally causing the spawned task to sleep until it sees that case upload record, in order to avoid a race condition. In environments that use CELERY_TASK_ALWAYS_EAGER, this process will time out, as the task will attempt to execute fully before the case upload record is created.

Instead, it appears that celery supports syntax to assign task IDs directly (see here). This avoids the same race condition, allows us to remove code, and lets this run with CELERY_TASK_ALWAYS_EAGER enabled.

Safety Assurance

Safety story

Verified this locally. I also verified that the task ID specified is indeed that task ID that is used within the task.

Automated test coverage

I'm not quite clear if automated tests are appropriate here. The main test I think is essentially a library test -- we want to verify that we can assign custom task IDs, and that's really celery's code, not ours.

QA Plan

No QA

Rollback instructions

  • This PR can be reverted after deploy with no further considerations

Labels & Review

  • Risk label is set correctly
  • The set of people pinged as reviewers is appropriate for the level of risk of the change

@mjriley mjriley added the product/invisible Change has no end-user visible impact label Jun 7, 2024
Copy link
Member

@dannyroberts dannyroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me, and I don't think I knew about the ability to set the task id manually like this. This does seem much simpler and cleaner.

My only suggestion is to do some manual testing on staging. The history of this change involves multiple reverts, which I take as a sign of unintuitive behavior, and then also sometimes asynchronous things can behave a little differently when there are actual machine-to-machine lagtimes and such. Just reading it though, this change seems logically correct to me.

Copy link
Contributor

@millerdev millerdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice simplification! Agreed with Danny about testing on staging.

@mjriley
Copy link
Contributor Author

mjriley commented Jul 12, 2024

Just commenting that I finally got around to testing this on staging, and everything works as expected

@mjriley mjriley merged commit 2c32147 into master Jul 12, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product/invisible Change has no end-user visible impact
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants