This PR implement the Parallel EVM engine

Co-authored-by: setunapo
Co-authored-by: sunny2022da
Co-authored-by: galaio
Co-authored-by: andyzhang2023

Fix an issue of incorrectly set the origin storage at parallel stateDB's
GetState(). Remove this code because it is already solved by lightCopy

PR: #2

rwset: support collect rwset from statedb;
mvstates: support export DAG;
dag: support travel all execution paths;
dag: refactor versioned TxDAG;
dag: support profile parallel execution path;
protocol: support to transfer TxDAG in NewBLock msg;

PR: #4

* fix several UT with racing issues

* fix incorrect nonce balance codehash issue

case: TestEIP1559 / TestDeleteThenCreate

* Fix ExecutionSpec tests

mainly root caused by balance not updated to dirty correctly.
also fix similar issue with nonce and codehash.

* fix TestBlockChain testcase issue

    TestBlockchain/ValidBlocks/bcStateTests/refundReset.json

Co-authored-by: Sunny <[email protected]>

* dag: add merge execute path method;
pevm: support dispatch with TxDAG;

* dag: add merge execute path method;
pevm: support dispatch with TxDAG;

* dag: clean code;

* statedb: fix some broken uts;

* pevm: support disable slot steal;

---------

Co-authored-by: galaio <[email protected]>

* fix several UT with racing issues

* fix incorrect nonce balance codehash issue

case: TestEIP1559 / TestDeleteThenCreate

* Fix ExecutionSpec tests

mainly root caused by balance not updated to dirty correctly.
also fix similar issue with nonce and codehash.

* fix TestBlockChain testcase issue

    TestBlockchain/ValidBlocks/bcStateTests/refundReset.json

* fix concurrent racing issue of state.accounts.

fix incorrect use of s.accountStorageParallelLock, it is designed
to be used for dirty/pending/original storages, not the accounts and
storages.

Use statedb.AccountMux and statedb.StorageMux for accounts/storages
lock.

* fix issue of DAOTransactions

handle the issue of updateObject of mainDB object touched by DAO transaction.

---------

Co-authored-by: Sunny <[email protected]>

There can be a issue that the object updated by mainDB.GetNonce etc is
obseleted.

The fix use statedb.getStateObjectNoUpdate to avoid touching the
stateObjects of mainDB.

Case: TestBlockchain/ValidBlocks/bcEIP1559/intrinsic.json

…ng; (bnb-chain#10)

* txdag: support txdag transfer in extra;

* txdag: support txdag transfer in extra;

---------

Co-authored-by: galaio <[email protected]>

txdag: record txdag metrics;

txdag: opt txdag flag name;

Co-authored-by: galaio <[email protected]>

…ain#12)

Fix 3 issues:
  - re-execution happens only to new version of baseDB to remove
    redundancy execution. And remove the retry with same baseIndex
    that is conflicted.
  - incorrect merge dirty objects in addrStateChangeInSlot, which
    cause incorrect data.root copied with obseleted stateDB, this
    fix handle the created, stateChanged and deleted object separately.
  - stateObject.GetCommitedState check mainDB of the object delete.

Co-authored-by: Sunny <[email protected]>

* pevm: support delay gas fee calculation;
txdag: check gas fee receiver;
tests: support PEVM+TxDAG UTs;

* txdag: skip some cost time operation;
tests: fix some broken UTs;

---------

Co-authored-by: galaio <[email protected]>

The originStorage will miss some loading in txn execution,do merge
rather than simple copy

This fix also use stateObject specific lock for storage update, rather
than the one in stateDB.

Co-authored-by: Sunny <[email protected]>

Co-authored-by: galaio <[email protected]>

txdag: opt rw record flag;

txdag: opt some logic;

Co-authored-by: galaio <[email protected]>

Co-authored-by: Sunny <[email protected]>

* txdag: opt some logic;

txdag: opt rw set collect logic;

* pevm: opt logs;

* txdag: opt txdag encoding, reduce rlp size;

---------

Co-authored-by: galaio <[email protected]>

* remove finalise

* fix: update maindb txIndex after merge slotDB

otherwise there can be issue that txIndex is load before the change in
mergeSlotDB.

* Fix: avoid update mainDB nonce in executeInSlot

It should use slotDB, otherwise it cause the stateObjects change in
mainDB, which cause racing issue.

* Fix: remove stateDB update during conflict check

stateDB.getState() will update the stateDB's stateObjects, which should
not be updated for the purpose of state read for conflict check.

---------

Co-authored-by: Sunny <[email protected]>

* txdag: add excluded flag;
mvstates: generate txdag with excluded flag;

* pevm: support txdag with excluded tx;

* blockchain: opt txdag file mode;

* pevm: fix dispatch bugs;

* pevm: opt txdag dispatch;

---------

Co-authored-by: galaio <[email protected]>

* pevm: opt slot trigger mechanism;

* txdag: opt execute stat;

* pevm: opt slot trigger mechanism;

* txdag: add txdag more validation logic;

---------

Co-authored-by: galaio <[email protected]>

Make the change into the merged mainDB instead of slotDB to avoid
the concurrency issue

Co-authored-by: Sunny <[email protected]>

* txdag: support new txdep resolve method;
pevm: avoid read txdag file when generating;

* pevm: support read txdag file in const size;

* txdag: reduce mem alloc and async resolve tx dependency;

---------

Co-authored-by: galaio <[email protected]>

)

* pevm: add some parallel tx metrics;

* pevm: opt read large txdag logic;

---------

Co-authored-by: galaio <[email protected]>

This change enable parallel execution when txs count >
parallelTxNum/2 + 2, with lower bound as 4, as the parallel execution
is slower than serial execution with single thread.

This change avoid the rewrite of reads cache in slotDB.

This change fix the issue that prevDestruct is not recorded correctly
and the stateObjectDestruct is not recorded in slotDB's createObject.
The incorrect record causes the GetState get obsoleted state as the
stateObjectDestruct is not correct.

* txDAG transfer

* set flag of txDAG transaction to 'no dependency'

* encode/decode txDAG data with ABI

* set enable flag for txdag

* set txDAG receiver to a special address

* remove invalid flags

---------

Co-authored-by: andyzhang2023 <[email protected]>

…bench tests; (bnb-chain#30)

* mvstate: using pending writes to accelerate txdag generation;

* txdag: test snappy compress ratio;

* txdag: add more bench tests;

---------

Co-authored-by: galaio <[email protected]>

This PR refine and clean up the code of PEVM

Co-authored-by: Sunny <[email protected]>

Co-authored-by: andyzhang2023 <[email protected]>

* txdag: support reset txdag reader when SetHead;

* txdag: clean some useless logs;

---------

Co-authored-by: galaio <[email protected]>

* txdag: fix system tx finalise issue;

* txdag: fix system tx finalise issue;

---------

Co-authored-by: galaio <[email protected]>

Co-authored-by: Sunny <[email protected]>

…ion; (bnb-chain#35)

* mvstates: fix async dep gen deadlock issue;
miner: support record sysytem tx rwset;

* miner: opt txdag enable checking;

---------

Co-authored-by: galaio <[email protected]>

* refine log to use debug for conflict detail

* fix: adjust parallelNum to use CpuNum-1 by default

---------

Co-authored-by: Sunny <[email protected]>

Co-authored-by: Sunny <[email protected]>

mvstates: opt resolve dep logic;

Co-authored-by: galaio <[email protected]>

Co-authored-by: galaio <[email protected]>

* fix: use slotdb pool

* fix UT of parallel slotDB

---------

Co-authored-by: Sunny <[email protected]>

Co-authored-by: Sunny <[email protected]>

* blockchain: avoid enable txdag generation when pevm is enabled;
mvstates: add timeout timer for async loop;

worker: change append TxDAG position;

worker: fix append TxDAG missing issue;

* blockchain: opt mining txdag generation logic;

---------

Co-authored-by: galaio <[email protected]>