Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Staging database restore can run concurrently with staging data refresh #5128

Closed
stacimc opened this issue Nov 1, 2024 · 0 comments · Fixed by #5155
Closed

Staging database restore can run concurrently with staging data refresh #5128

stacimc opened this issue Nov 1, 2024 · 0 comments · Fixed by #5155
Assignees
@krysal @stacimc
Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@stacimc
Copy link
Collaborator

stacimc commented Nov 1, 2024

Description

If the staging database restore runs concurrently with a staging data refresh, it wipes out the DB while it is being acted upon and causes everything to fail (noisily).

Something must not be working in the existing concurrency checks.

Workaround: disable the staging_database_restore DAG while testing data refreshes in staging (hence high priority rather than critical).
@stacimc stacimc added 🛠 goal: fix Bug fix 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Nov 1, 2024
@stacimc stacimc self-assigned this Nov 1, 2024
@openverse-bot openverse-bot moved this to 📋 Backlog in Openverse Backlog Nov 1, 2024
@openverse-bot openverse-bot moved this from 📋 Backlog to 📅 To Do in Openverse Backlog Nov 1, 2024
@krysal krysal added the 💻 aspect: code Concerns the software code in the repository label Nov 8, 2024
@krysal krysal mentioned this issue Nov 8, 2024
Merged
8 tasks
@openverse-bot openverse-bot moved this from 📅 To Do to 🏗 In Progress in Openverse Backlog Nov 8, 2024
@krysal krysal self-assigned this Nov 11, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to ✅ Done in Openverse Backlog Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@krysal krysal

@stacimc stacimc

Labels
💻 aspect: code Concerns the software code in the repository 🛠 goal: fix Bug fix 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Openverse Backlog
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Append concurrency tag to prevent issues between DAGs in staging WordPress/openverse
2 participants
@krysal @stacimc