Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug/1588 change snapshot hash computation #1787

Merged
merged 9 commits into from
Jan 18, 2024

Conversation

olehnikolaiev
Copy link
Contributor

@olehnikolaiev olehnikolaiev commented Jan 17, 2024

fixes #1588

do not force leveldb to store all the key-value pairs in OS buffer
instead iterate over the database batch by batch to decrease RAM usage during snapshot hash calculation

tested on devnet:

  • update one node to the version with fix
  • run load tests for the long time. you will see that RAM and swap usage is less for the updated node ( @oleksandrSydorenkoJ )
  • non updated nodes will eventually fail with out of memory error, while updated node will stay healthy

@olehnikolaiev olehnikolaiev changed the title Bug/1640 change snapshot hash computation Bug/1588 change snapshot hash computation Jan 17, 2024
@olehnikolaiev olehnikolaiev linked an issue Jan 17, 2024 that may be closed by this pull request
Copy link

codecov bot commented Jan 17, 2024

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (678f59b) 46.02% compared to head (a79c45d) 45.68%.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           v3.18.0    #1787      +/-   ##
===========================================
- Coverage    46.02%   45.68%   -0.35%     
===========================================
  Files          358      358              
  Lines        51820    51878      +58     
===========================================
- Hits         23852    23701     -151     
- Misses       27968    28177     +209     

dimalit
dimalit previously approved these changes Jan 17, 2024
dimalit
dimalit previously approved these changes Jan 18, 2024
@olehnikolaiev olehnikolaiev requested a review from dimalit January 18, 2024 13:06
@DmytroNazarenko DmytroNazarenko merged commit 46bbdbd into v3.18.0 Jan 18, 2024
7 of 8 checks passed
@DmytroNazarenko DmytroNazarenko deleted the bug/1640-change-snapshot-hash-computation branch January 18, 2024 15:10
@github-actions github-actions bot locked and limited conversation to collaborators Jan 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RAM growth while snapshot hash computation
4 participants