Allow restoration of Greenplum backups using a newer version of gpbackup #8
Conversation
Disable version compatibility checks
Fix error invalid go version '1.21.4': must match format 1.23
revert previous change
Reinstate version checks but change to warning
remove github.com/pkg/errors
|
|
|
It appears that the unit tests will need to be rewritten in order for the tests to pass as they specifically check for Cloudberry version compatibility only and make no consideration of restoring Greenplum data to Cloudberry |
|
@my-ship-it, @xanadu-dev & @Terry1504 Do you have an idea when this PR will make some progress. We have a user in the community that is using gprestore with this fix. It would be good to get the test issues addressed and this merged. |
I understand that simply lowering the warning level without stopping the user's backup and restore operations is not compliant. You can refer to the test cases of EnsureBackupVersionCompatibility and EnsureDatabaseVersionCompatibility in the code. In this way, data consistency from a higher version to a lower version cannot be guaranteed. |
This PR was simply a workaround for the situation where the checks fail the restore because the Cloudberry Gprestore version has a lower version number than the Greenplum Gpbackup version. This approach is not really viable going forward as I can imagine it could cause incompatibility issues. However there still needs to be some way to migrate from Greenplum to Cloudberry using backup/restore. One simple solution would be if the Cloudberry Gpbackup/Gprestore versioning started at version 2.X - then if trying to restore backup data from Greenplum to Cloudberry during a migration then checks would succeed as the Cloudberry version is later than the Greenplum version 1.X. Even if the process is not fully supported then it provides a means of migration to CloudberryDB - without making such a change I cannot see any practical way of migrating and we will remain using Greenplum |
| @@ -298,8 +297,8 @@ func EnsureBackupVersionCompatibility(backupVersion string, restoreVersion strin | |||
| restoreSemVer, err := semver.Make(restoreVersion) | |||
| gplog.FatalOnError(err) | |||
| if backupSemVer.GT(restoreSemVer) { | |||
| gplog.Fatal(errors.Errorf("gprestore %s cannot restore a backup taken with gpbackup %s; please use gprestore %s or later.", | |||
| restoreVersion, backupVersion, backupVersion), "") | |||
| gplog.Warn("gprestore %s cannot restore a backup taken with gpbackup %s; please use gprestore %s or later.", | |||
There was a problem hiding this comment.
Generally speaking, gpbackup is mainly used for database backup, not for upgrading, because different versions may have potential data compatibility issues.
If upgrading is needed, it is recommended to use other approaches.
There was a problem hiding this comment.
The problem I have is that I have not yet found any reference to "other approaches" for the greenplum to cloudberry transition and in my case this approach worked and enabled me to continue evaluation of cloudberry-db.
I can see why this PR cannot be accepted but the simplest solution I suggested earlier would be to change the versioning so that gpbackup could be used without any changes for importing greenplum data to cloudberry
One simple solution would be if the Cloudberry Gpbackup/Gprestore versioning started at version 2.X - then if trying to restore backup data from Greenplum to Cloudberry during a migration then checks would succeed as the Cloudberry version is later than the Greenplum version 1.X.
Even if the process is not fully supported then it provides a means of migration to CloudberryDB - without making such a change I cannot see any practical way of migrating and we will remain using Greenplum
Is this the recommended tool for migration ?
https://github.com/cloudberry-contrib/cbcopy
Enable Cloudberry gpbackup restore process to restore backups from Greenplum which have been taken using a newer version of gpbackup by changing functions EnsureBackupVersionCompatibility and EnsureDatabaseVersionCompatibility functions so that a warning is thrown rather than an error