Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Admin restarts skaled container instead of recreating after node rotation #976

Closed
yatsunastya opened this issue Sep 14, 2023 · 3 comments · Fixed by skalenetwork/skaled#1649
Assignees
Labels

Comments

@yatsunastya
Copy link

yatsunastya commented Sep 14, 2023

It's not reproducible every node rotation

Describe the bug
Each n seconds admin does set of checks and on the base of its results admin chooses monitor type.
During node rotation, after finish TS admin recreates skaled containers on all staying nodes.

In this case, admin got the results of its checks and chose Regular Monitor. And when skaled exited at finish TS, admin just restarted it, as it's expected during Regular Monitor.
As a result skaled container and schain config weren't recreated.

[2023-09-13 16:49:32,103 INFO][18679][free-basic-rosy-SKALE][T_0] - core.schains.monitor.containers:94 - sChain free-basic-rosy-SKALE: restarting container
[2023-09-13 16:49:32,104 INFO][18679][free-basic-rosy-SKALE][T_0] - core.schains.runner:167 - 
====================================================================================================
Restarting container...
Container name: skale_schain_free-basic-rosy-SKALE
====================================================================================================

The problem is that skaled exited just after recording checks results, so the results for admin and real state are different.

Version:
Admin: 2.5.1-develop.4

Logs:
admin_restarts.txt

@DmytroNazarenko
Copy link
Contributor

The problem is that skaled finished with exit code 1 during node rotation, in previous release it was finishing with code 0

@DmytroNazarenko
Copy link
Contributor

skaled: 3.17.0-beta.9

@DmytroNazarenko DmytroNazarenko moved this from Ready For Release Candidate to Merged To Release Candidate in SKALE Engineering 🚀 Sep 16, 2023
@EvgeniyZZ EvgeniyZZ moved this from Merged To Release Candidate to QA in SKALE Engineering 🚀 Oct 4, 2023
@yatsunastya
Copy link
Author

checked ✅
skaled: 3.17.1

Now, after rotation, skaled on leaving node exits with 0 exit code, and the problem wasn't reproduced.

@EvgeniyZZ EvgeniyZZ moved this from QA to Done in SKALE Engineering 🚀 Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants