Consolidate TxLimits

[nfrisby]

Closes #1177.

Beyond that Issue, this PR also changes the mempool's definition of full. The mempool will accepte transactions until the tx chosen to next enter the mempool would cause any component of the capacity vector (byte size, exunit steps, exunit mems, ref script byte size) to exceed its limit. The default capacity is chosen as twice the capacity of a block. Such a capacity can admit more txs than could fit into two back-to-back blocks, but no single component (eg ExUnits) of the mempool's contents' size could exceed what could fit in two blocks.

The mempool capacity has two consequences.

• It implements latency hiding: txs can cover the network a little in advance, so that two blocks on a chain whose slots are very close together could still both be full (if there are enough submitted txs).
• It bounds the potential burst of CPU load necessary to fill an empty mempool, which otherwise could be a DoS vector.

Also closes #1168 by superseding the "ad-hoc" mempool reference scripts capacity limit.

This PR also moves the logic for choosing which prefix of the mempool to put in the block out of the forging functions and into the NodeKernel forging thread --- the mempool snapshot is the perfect place to do that, since tx measurements (each using the correct ledger state) are already determined.

[nfrisby]

Phew! I think it's compiling without warnings and I'm optimistic the tests will pass on CI.

But the commit history is a total mess, and some new comments are missing, at the very least.

[fraser-iohk]

I also have some questions about what the TxMeasure types should look like, especially in the context of #1178, but they're probably better suited for discussion on a call

[nfrisby]

I rebased

[nfrisby]

The next step for this PR is to merge #1182. Eventually I'll rebase this one on top of that.

[nfrisby]

I don't know if we want to merge this before the Chang fork, so this remains in Draft. I think that's the only reason, though. We need to consider the benchmark numbers.

  10000 transactions
    setup mempool:             FAIL (0.17s)
      156  μs ±  11 μs, 166 KB allocated, 132 B  copied, 6.0 MB peak memory, 179% more than baseline
      Use -p '$0=="Just adding.10000 transactions.setup mempool"' to rerun this test only.
    setup mempool + benchmark: OK (0.73s)
      23.1 ms ± 1.0 ms,  68 MB allocated, 4.0 MB copied,  11 MB peak memory, 19% more than baseline
    test:                      OK (0.02s)
    txs length:                OK
  20000 transactions
    setup mempool:             FAIL (0.19s)
      330  μs ±  27 μs, 327 KB allocated, 252 B  copied,  13 MB peak memory, 176% more than baseline
      Use -p '$0=="Just adding.20000 transactions.setup mempool"' to rerun this test only.
    setup mempool + benchmark: OK (0.13s)
      46.0 ms ± 4.5 ms, 139 MB allocated, 5.9 MB copied,  15 MB peak memory, 17% more than baseline
    test:                      OK (0.06s)
    txs length:                OK

I'm not surprised the benchmark itself is a bit slower: the arithmetic in the mempool are now routed through dictionaries, since the size (it's just ByteSize in this test) is now a polymorphic field in the finger tree measure instead of a strict monomorphic Word32.

I'm pretty sure the setup is ~3x slower just because I'm using foldMap at type ByteSize instead of sum $ fmap at a known type of Word32 --- the total size calculation is done during the setup. (foldMap' wins some of that back, but not all 🤔). Moreover, Word32 has more optimized instances, eg (in particular, the Generics-based InstantiatedAt stuff has not been performance tuned). Something to consider, if we see meaningful degradation in the real system.

[nfrisby]

OK. I added a few commits, after chatting with Esgen.

• consensus-cardano: use ConwayMeasure in Babbage
• TOSQUASH fixup overflow workarounds and their explanations
• consensus: use Natural for ByteSize

The last one supersedes the middle one. The last's commit message already asserting the performance penalty is acceptable, but this deserves some due diligence.

I have a relatively simple idea for that: use (a trivial new) db-analyser to emit the (new) TxMeasure for two recent epochs from mainnet. Build a benchmark that simply adds and removes "transactions" that are nothing except one of those recorded measurements (in the same order as observed on mainnet). This can be done at the TxSeq level, and so won't require the menagerie of blk instances.

Such a benchmark will yield a characteristic cost (eg "total runtime / number of txs"), which I'm hoping/expecting will be quite smaller than the (comparably averaged) cost of fully validating a tx.

[nfrisby]

I haven't yet reorganized the commits into a nice sequence yet, but review can commence --- I will ensure the final diff matches that of the current tip (ie fe778a3)

[nfrisby]

I'm now attempting to split the monolithic commit. I consider it to have the following components.

• forging thread no longer splits the given list, see snapshotTake instead

• refactor the TxLimits class and CanHardFork

• removed the Babbage mempool ref scripts hotfix

• remove the block size override BoundedMeasure

• multi-dimensional tx sizes in the mempool itself, careful with overflow and deadlock DoS, and supplanting the txSize argument

I'm hoping I can tease out a separate commit for at least a few of those.

Edit: I pulled out a couple. Especially the one that removes Mempool.getTxSize is nice to isolate. The final commit is quite large still, but its changes are not easy to disentangle.

[amesgen]

Excellent 🎉

Only minor comments

[nfrisby]

Some collisions with ouroboros-network (they made SizeInBytes a newtype instead of a type synonym) during the rebase, but easy enough (disclaimer: I haven't ran the tests locally).

[nfrisby]

The mempool-bench is failing.

Just adding
  10000 transactions
    setup mempool:             FAIL (0.25s)
      1.75 μs ± 153 ns, 6.2 KB allocated,   3 B  copied, 5.0 MB peak memory, 231% more than baseline
      Use -p '$0=="Just adding.10000 transactions.setup mempool"' to rerun this test only.
    setup mempool + benchmark: OK (0.37s)
      24.4 ms ± 741 μs,  68 MB allocated, 4.0 MB copied,  11 MB peak memory, 31% more than baseline
    test:                      OK (0.03s)
    cmds length:               OK
  20000 transactions
    setup mempool:             FAIL (0.28s)
      1.80 μs ±  99 ns, 6.4 KB allocated,   6 B  copied,  13 MB peak memory, 239% more than baseline
      Use -p '$0=="Just adding.20000 transactions.setup mempool"' to rerun this test only.
    setup mempool + benchmark: OK (0.15s)
      48.7 ms ± 3.0 ms, 141 MB allocated, 5.9 MB copied,  15 MB peak memory, 25% more than baseline
    test:                      OK (0.06s)
    cmds length:               OK

Joris helpfully shared the numbers from the latest baseline file:

Just adding
  10000 transactions
    setup mempool:             OK (0.16s)
      528  ns ±  42 ns, 2.7 KB allocated,   1 B  copied, 5.0 MB peak memory
    setup mempool + benchmark: OK (0.28s)
      18.5 ms ± 1.1 ms,  57 MB allocated, 3.4 MB copied, 9.0 MB peak memory
    test:                      OK (0.02s)
    cmds length:               OK
  20000 transactions
    setup mempool:             OK (0.19s)
      530  ns ±  46 ns, 2.7 KB allocated,   1 B  copied,  12 MB peak memory
    setup mempool + benchmark: OK (1.22s)
      38.6 ms ± 521 μs, 119 MB allocated, 7.6 MB copied,  24 MB peak memory
    test:                      OK (0.03s)
    cmds length:               OK

The failing case is the setup step, which increases from 500 nanoseconds to 1750 nanoseconds.

It is very plausible that the additional calculation in computeMempoolCapacity explains this: it use to be * 2 at known type Word32 and is now a call to some small amount of arithmetic and stimes involving a couple dictionaries. It's harmless for the behavior of real node (setup happens once per execution, and it's still microseconds).

So I'm merging despite these failures --- they are not required "checks" and this benchmark is the only FAIL within the checks.