Skip to content

Latest commit

 

History

History
78 lines (50 loc) · 2.78 KB

switchover.md

File metadata and controls

78 lines (50 loc) · 2.78 KB
Raw

Switchover Guide for GoAlert

Switchover (SWO) is a feature that allows a live system to switch from one database to another safely and with little to no user impact.

This guide provides a quick and easy-to-follow set of steps for performing a database switchover in GoAlert.

Steps to Perform Switchover

  1. Preparation: Configure all GoAlert instances with the --db-url-next flag (or using GOALERT_DB_URL_NEXT environment variable). Ensure that every instance is started or restarted with this configuration.

    goalert --db-url=<old-db-url> --db-url-next=<new-db-url>

    OR using environment variables:

    export GOALERT_DB_URL=<old-db-url>
export GOALERT_DB_URL_NEXT=<new-db-url>

  2. Check Configuration in UI:

    • Navigate to the Admin section in the left sidebar of the UI.
    • Click on the Switchover page.
    • Ensure that all instances of GoAlert (displayed as Nodes) have a green checkmark next to Config Valid?.

  3. Initialize Switchover:

    • While still on the Switchover page, click the RESET button. This should initiate configuration and other checks.
    • Make sure everything looks good and is validated.

  4. Execute Switchover:

    • Click the EXECUTE button to perform the database switch.
    • If the operation fails (it will fail safely), click RESET and then EXECUTE again.

  5. Post-Switchover Configuration:

    • Once the switchover is successful, reconfigure all instances to use only --db-url, now pointing to the NEW database URL.
    • Remove the --db-url-next flag or unset GOALERT_DB_URL_NEXT.
    goalert --db-url=<new-db-url>

    OR using environment variables:

    export GOALERT_DB_URL=<new-db-url>
unset GOALERT_DB_URL_NEXT

Rollback Procedures

In case you encounter issues during the switchover or decide to cancel the operation, you can do so safely by following these rollback steps:

  1. Cancel Switchover in UI:

    • Navigate to the Switchover page under the Admin section in the UI.
    • Click the RESET button. This will cancel the ongoing switchover process and restore the original database configuration.

  2. Reconfigure Instances:

    • Redeploy or restart your GoAlert instances with the original --db-url flag, while removing the --db-url-next flag or the corresponding environment variable GOALERT_DB_URL_NEXT.
    goalert --db-url=<old-db-url>

    OR using environment variables:

    export GOALERT_DB_URL=<old-db-url>
unset GOALERT_DB_URL_NEXT

Note: After a successful switchover, the old database will be marked as obsolete. GoAlert instances will refuse to start if configured to use this old database. Therefore, rollback after a successful switchover is not possible without administrative intervention.