skaled imported block 5866944 before block 5866938

[olehnikolaiev]

after the crash skaled restarted and started to download missing blocks. while importing them skaled imported block 5866944 before block 5866938

logs from the crashed node :
Discover search [2024-10-03T15_41_21.547+01_00].csv

logs from the healthy node:
Discover search [2024-10-03T17_30_21.984+01_00].csv

[olehnikolaiev]

similar situation happened after another crash: skaled imported blocks up to #13899435 but after crash skaled reported that Last block in skaled: 13899415. however, consensus reported that Last committed block in consensus:13899435. then skaled started importing block #13899416 but got an error TX EXECUTION WARNING: expected 7 transaction(s) to pass, when 6 passed with success,0 expected to run and pass. it lead to the incorrect blockchain state for that node

see the logs attached

Discover search [2024-10-17T12_40_19.773+01_00].csv

[olehnikolaiev]

block 16179075 was imported before block 16179074
Discover search [2024-10-17T15_29_28.489+01_00].csv

[dimalit]

[58:main] [error] 16179073:!Exception: Schain:processCommittedBlock
[58:main] [error] 16179073: !Caused by: Schain:State check failed::currentTime >= lastCommitTimeMs /home/s5/actions-runner-1/_work/skaled/skaled/libconsensus/chains/SchainGettersSetters.cpp:314

These variables are managed in the following way:

    auto currentTime = Time::getCurrentTimeMs();
    CHECK_STATE( currentTime >= lastCommitTimeMs );
    lastCommitTimeMs = currentTime;

So, the only possible way to have this broken, is to have this function executed in multiple threads. This can happen in case of simultaneous work of catch-up and consensus.

[PolinaKiporenko]

retest after state locking