Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Scheduling Spec Goals Delete (for develop) #1436

Merged
merged 5 commits into from
May 9, 2024

Conversation

Mythicaeda
Copy link
Contributor

  • Tickets addressed: Hotfix
  • Review: By commit
  • Merge strategy: Merge (no squash)

Description

Sister PR of #1435.

PR Description from 1435:

These PRs fix two bugs with our scheduling specifications: 1) The migration for Versioning Scheduling Goals did not update the `when` condition of the `on update` trigger for scheduling specifications, leading to the stack depth unexpectedly exploding when trying to update a spec 38+ goals (the threshold may be lower, but I did not look into this). 2) The function used for `on delete` did not properly handle bulk deletes (as in a delete statement that deletes multiple rows). This function and its trigger have been updated to operate "per-statement" and go through all the rows deleted in reverse priority order. This ensures that

Independently of these bugs, I also put a depth check on the increment spec update when the goals subtable is updated trigger, as there's no reason for us to update the revision when a trigger changes a goal's priority.

A simple example of the second bug follows:
Let's say we have the following specification. Specification are being expressed as an array ordered by priority and "tailing" priorities will be expressed as null in order to avoid confusion caused by the array changing size mid-walkthrough.
[ Goal 1, Goal 2, Goal 3, Goal 4, Goal 5, Goal 6 ]

Now let us delete Goal 1, Goal 2, and Goal 5. The expected output for this action is[Goal 3, Goal 4, Goal 6, null, null, null]

Under the old logic, the following would happen:

  1. Goal 1, Goal 2, and Goal 5 are deleted: [null, null, Goal 3, Goal 4, null, Goal 6]
  2. The after delete trigger for Goal 1 is processed. As Goal 3, Goal 4, and Goal 6 have a strictly higher priority than Goal 1, their priorities are reduced by 1: [null, Goal 3, Goal 4, null, Goal 6, null]
  3. The after delete trigger for Goal 2 is processed. As Goal 4 and Goal 6 have a strictly higher priority than Goal 2, their priorities are reduced by 1: [null, Goal 3 and Goal 4, null, Goal 6, null, null]
  4. The after delete trigger for Goal 5 is processed. As no Goals have a strictly higher priority than Goal 5, no rows are updated: [null, Goal 3 and Goal 4, null, Goal 6, null, null]
  5. An exception is raised, as Goal 3 and Goal 4 cannot share a priority.

Note that preserving the old per-row function but instead reducing goals with a priority greater than or equal to the current goal's would not have been sufficient, as it would lead to gaps in priorities (as well as conflicting priorities in different setups). For example, in the above example, Goal 6 has been placed at priority 4 before we process Goal 5's deletion, meaning it will not be moved to priority 3, even though that's where we would expect it to be at the end of the transaction, leaving a gap at priority 3.

Under the new logic, the following occurs:

  1. Goal 1, Goal 2, and Goal 5 are deleted: [null, null, Goal 3, Goal 4, null, Goal 6]
  2. The after delete for step 1 begins. Goal 5, as the deleted Goal with the highest priority, is processed first: [null, null, Goal 3, Goal 4, Goal 6, null]
  3. Goal 2, as the deleted Goal with the second-highest priority, is processed next: [null, Goal 3, Goal 4, Goal 6, null, null]
  4. Goal 1, as the deleted Goal with the lowest priority, is processed last: [Goal 3, Goal 4, Goal 6, null, null, null]
  5. The after delete for step 1 completes.

The output in this case is exactly as expected, as by processing in reverse priority order, we guarantee that all goals with a strictly higher priority than that of the deleted goal we're processing is still at a strictly higher priority than that deleted goal, even if that priority has changed.

Verification

DB Tests were added to test bulk deleting from the both types of specs in three different scenarios: a chunk out of the middle, only the middle left, and many alternating rows. Against develop, these tests are expected to fail.

Documentation

As said in 1435, we need to document that PGCMP does not catch when conditions on triggers so that viewers are aware to keep their eye out for it.

Future work

As said in 1435, looking into extending PGCMP to implement the when condition trigger match.

Mythicaeda added 4 commits May 2, 2024 12:01
By incrementing through the deleted goals in reverse priority order, we guarantee that all the remaining goals that had a higher priority than the deleted goal before the delete will still be at a higher priority when we process the deleted row.

- Add trigger depth restriction on `increment_revision_on_goal_update`
@Mythicaeda Mythicaeda added fix A bug fix scheduling Anything related to the scheduling domain database Anything related to the database labels May 7, 2024
@Mythicaeda Mythicaeda added this to the FY24 Q3 - Bug Fixes milestone May 7, 2024
@Mythicaeda Mythicaeda requested review from JoelCourtney and skovati May 7, 2024 00:40
@Mythicaeda Mythicaeda self-assigned this May 7, 2024
@Mythicaeda Mythicaeda requested a review from a team as a code owner May 7, 2024 00:40
Copy link
Contributor

@skovati skovati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests look good to me!

.github/workflows/pgcmp.yml Show resolved Hide resolved
@Mythicaeda Mythicaeda merged commit 92d0e0d into develop May 9, 2024
10 checks passed
@Mythicaeda Mythicaeda deleted the fix/fix-delete-trigger branch May 9, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
database Anything related to the database fix A bug fix scheduling Anything related to the scheduling domain
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants