Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add runbook to restore stackgres backup #9860

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

xin-hedera
Copy link
Collaborator

@xin-hedera xin-hedera commented Dec 2, 2024

Description:

This PR adds a script to restore stackgres backup

  • Add a script to restore citus from a stackgres sharded backup with ZFS snapshots
  • Add a simple readme

Related issue(s):

Fixes #9577

Notes for reviewer:

Tested the following scenario:

  • simple case, restore the most recent SGShardedBackup
  • restore an older SGShardedBackup, any backups after the restore target are removed by the script
  • restore an SGShardedBackup in which the coordinator data is backed up from coord-1 instead of the default coordinator primary coord-0

Overall rolling back a ZFS volume to a snapshot and recover the saved WAL files are fast, however it make take longer if the data diff size is larger between the snapshot and the current volume.

The more time consuming part is re-initializing the coordinator replica from the primary, it took about 5 minutes for testnet with 5GB coordinator data.

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

@xin-hedera xin-hedera added this to the 0.120.0 milestone Dec 2, 2024
@xin-hedera xin-hedera self-assigned this Dec 2, 2024
@xin-hedera xin-hedera requested a review from a team as a code owner December 2, 2024 14:53
@xin-hedera xin-hedera linked an issue Dec 2, 2024 that may be closed by this pull request
@xin-hedera xin-hedera added the enhancement Type: New feature label Dec 2, 2024
Copy link

codecov bot commented Dec 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.26%. Comparing base (3d945d9) to head (87619b3).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9860      +/-   ##
============================================
- Coverage     92.27%   92.26%   -0.01%     
  Complexity     7771     7771              
============================================
  Files           951      951              
  Lines         32472    32472              
  Branches       4118     4118              
============================================
- Hits          29962    29961       -1     
  Misses         1546     1546              
- Partials        964      965       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

docs/configuration.md Outdated Show resolved Hide resolved
Comment on lines +27 to +28
read -p "Continue? (Y/N): " confirm && [[ $confirm == [yY] || $confirm == [yY][eE][sS] ]] && return || \
{ [[ -n "$confirm" ]] && exit 1 || true; }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change the function to ignore empty input

jnels124
jnels124 previously approved these changes Dec 3, 2024
Copy link
Contributor

@jnels124 jnels124 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

done
}
CURRENT_NAMESPACE=$(kubectl config view --minify --output 'jsonpath={..namespace}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add to requirements in the .md?

steven-sheehy
steven-sheehy previously approved these changes Dec 3, 2024
Copy link
Member

@steven-sheehy steven-sheehy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

sonarcloud bot commented Dec 3, 2024

Copy link
Contributor

@edwin-greene edwin-greene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Type: New feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restore Citus Backup
4 participants