Skip to content

Commit

Permalink
Merge bitcoin#31534: coins: warn on shutdown for big UTXO set flushes
Browse files Browse the repository at this point in the history
5709718 coins: warn on shutdown for big UTXO set flushes (Lőrinc)

Pull request description:

  Split out of bitcoin#30611 (comment)

  Setting a large `-dbcache` size postpones the index writes until the coins cache size exceeds the specified limit. This causes the final flush after manual termination to seemingly hang forever (e.g. tens of minutes for 20 GiB); Now that the `dbcache` upper cap has been lifted, this will become even more apparent, so a warning will be shown when large UTXO sets are flushed (currently >1 GiB), such as:
  > 2024-12-18T18:25:03Z Flushed fee estimates to fee_estimates.dat.
  > 2024-12-18T18:25:03Z [warning] Flushing large (1 GiB) UTXO set to disk, it may take several minutes
  > 2024-12-18T18:25:09Z Shutdown: done

  ---

  You can reproduce it by starting `bitcoind` with a large `-dbcache`:
  > mkdir demo  && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j$(nproc) && build/src/bitcoind -datadir=demo **-dbcache=10000**

  Waiting until the used memory is over 1 GiB
  > 2024-12-18T18:25:02Z UpdateTip: [...] progress=0.069009 cache=**1181.1MiB**(8827981txo)

  And cancelling the process from the terminal:
  > ^C2024-12-18T18:25:03Z tor: Thread interrupt
  > [...]
  > 2024-12-18T18:25:03Z **[warning] Flushing large (1 GiB) UTXO set to disk, it may take several minutes*

ACKs for top commit:
  sipa:
    utACK 5709718
  tdb3:
    re ACK 5709718
  1440000bytes:
    ACK bitcoin@5709718
  danielabrozzoni:
    tACK 5709718

Tree-SHA512: 608cf797de788501ccb2986508c155f5660c5f6f7a414524bfcc2820cfa9ebe3da558d13f2317d1f121a82d49ffe1e711a1152c743c22dab9f9807363f4ed8d5
  • Loading branch information
ryanofsky committed Dec 27, 2024
2 parents f95fb79 + 5709718 commit 9355578
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions src/validation.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ using node::CBlockIndexHeightOnlyComparator;
using node::CBlockIndexWorkComparator;
using node::SnapshotMetadata;

/** Size threshold for warning about slow UTXO set flush to disk. */
static constexpr size_t WARN_FLUSH_COINS_SIZE = 1 << 30; // 1 GiB
/** Time to wait between writing blocks/block index to disk. */
static constexpr std::chrono::hours DATABASE_WRITE_INTERVAL{1};
/** Time to wait between flushing chainstate to disk. */
Expand Down Expand Up @@ -2929,8 +2931,9 @@ bool Chainstate::FlushStateToDisk(
}
// Flush best chain related state. This can only be done if the blocks / block index write was also done.
if (fDoFullFlush && !CoinsTip().GetBestBlock().IsNull()) {
LOG_TIME_MILLIS_WITH_CATEGORY(strprintf("write coins cache to disk (%d coins, %.2fkB)",
coins_count, coins_mem_usage / 1000), BCLog::BENCH);
if (coins_mem_usage >= WARN_FLUSH_COINS_SIZE) LogWarning("Flushing large (%d GiB) UTXO set to disk, it may take several minutes", coins_mem_usage >> 30);
LOG_TIME_MILLIS_WITH_CATEGORY(strprintf("write coins cache to disk (%d coins, %.2fKiB)",
coins_count, coins_mem_usage >> 10), BCLog::BENCH);

// Typical Coin structures on disk are around 48 bytes in size.
// Pushing a new one to the database can cause it to be written
Expand Down

0 comments on commit 9355578

Please sign in to comment.