[Feature] Support restoring from a cluster snapshot for shared-data mode (part 1) #53861
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![starrocks-cr[bot]](https://avatars.githubusercontent.com/in/404334?s=60&v=4){kind=small}
fe/fe-core/src/main/java/com/starrocks/common/RestoreSnapshotConfig.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/com/starrocks/server/RestoreSnapshotMgr.java
Outdated
Show resolved
Hide resolved
xiangguangyxg
force-pushed
the
restore_snapshot_yaml
branch
2 times, most recently
from
cc323ba to
3573838
Compare
7 tasks
xiangguangyxg
force-pushed
the
restore_snapshot_yaml
branch
2 times, most recently
from
c7b9e88 to
7d13b75
Compare
srlch
reviewed
Dec 13, 2024
fe/fe-core/src/main/java/com/starrocks/server/RestoreSnapshotMgr.java
Outdated
Show resolved
Hide resolved
xiangguangyxg
force-pushed
the
restore_snapshot_yaml
branch
from
7d13b75 to
4065fba
Compare
…part 1) Signed-off-by: xiangguangyxg <[email protected]>
xiangguangyxg
force-pushed
the
restore_snapshot_yaml
branch
from
4065fba to
dac7a9f
Compare
|
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[FE Incremental Coverage Report]✅ pass : 109 / 126 (86.51%) file detail
|
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
xiangguangyxg
changed the title
gengjun-git
reviewed
Dec 18, 2024
fe/fe-core/src/main/java/com/starrocks/lake/snapshot/RestoreClusterSnapshotMgr.java
Show resolved
Hide resolved
gengjun-git
reviewed
Dec 18, 2024
fe/fe-core/src/main/java/com/starrocks/lake/snapshot/RestoreClusterSnapshotMgr.java
Show resolved
Hide resolved
HangyuanLiu
reviewed
Dec 18, 2024
| @@ -0,0 +1,39 @@ | |||
| # do not include leader fe | |||
| frontends: | |||
There was a problem hiding this comment.
FE configure file is xml or json，why invole a new config type ?
There was a problem hiding this comment.
using yaml is designed by pm ...
| FOLLOWER, | ||
| OBSERVER; | ||
|
|
||
| @JsonCreator |
There was a problem hiding this comment.
Can we use json? This way, we can unify the development to GSON and reduce the types of configuration for users.
There was a problem hiding this comment.
json do not support comment and is suitable for machine parsing but not for human reading, it is not suitable for configuration file.
yaml support comment and suitable for both machine parsing and human reading, it is a better new generation configuration file.
by the way, yaml can be handled by Jackson, which is used in many places in FE.
gengjun-git
approved these changes
Dec 18, 2024
srlch
approved these changes
Dec 18, 2024
HangyuanLiu
approved these changes
Dec 18, 2024
7 tasks
|
@mergify backport branch-3.4 |
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
Dec 19, 2024
…ode (part 1) (#53861) Signed-off-by: xiangguangyxg <[email protected]> (cherry picked from commit 969593c)
18 tasks
wanpengfei-git
pushed a commit
that referenced
this pull request
Dec 19, 2024
…ode (part 1) (backport #53861) (#54074) Co-authored-by: xiangguangyxg <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
To support disaster recovery for shared-data mode.
What I'm doing:
Introduce a config file
cluster_snapshot.yamlfor FE.Put the image file of a cluster snapshot in
fe/meta/imageof the leader FE and clear thefe/meta/bdb,than use
./bin/start_fe.sh --cluster_snapshot --daemonto start to restore from the cluster snapshot.Other FE nodes use
./bin/start_fe.sh --helper <leader_ip>:<leader_edit_log_port> --daemonto start.What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: