Improve data destruction in EVM

[DmytroNazarenko]

There was an ineffective algorithm for data destruction in EVM that got fixed.

We iterate over LevelDB keys belonging to the account that has been destroyed and remove them

See also

• https://github.com/skalenetwork/internal-support/issues/556

Tasks

Preview Give feedback

1. Create patch timestamp for StorageDestructionPatch skale-node#527
   1 of 1
   Technical debt release:2.2 +2
   

[kladkogex]

Assigning to myself since I am already working on making data cleanup more effective

[DmytroNazarenko]

Check after levelDB update

[oleksandrSydorenkoJ]

STR: TC-1288

[DmytroNazarenko]

3.17.0-beta.1

[oleksandrSydorenkoJ]

Verified on Regression network
schain:3.17.0-beta.5

20 MB = 7 seconds
100 MB = 29 seconds

skaled_selfdestruct.log

[DmytroNazarenko]

In this version we have switched on back the algorithm of destruction of the contracts' state, that is called by SELFDESTRUCT opcode. The algorithm is enabled from the storageDestructionPatchTimestamp. In the new version version of algorithm we iterate over LevelDB keys belonging to the account that has been destroyed and remove them. In the end all data in leveldb belonging to this account should be destroyed and the disk space should be freed accordingly

For the future investigation: #1644

[oleksandrSydorenkoJ]

Self destruct function clears only contract's address, but not contract data.
And example 2.1.1 release, where selfdestruct function doesn't work at all

free-basic-rosy-SKALE
0xf97fA6B3c51D0ddfB83dE0e5fa5958528f3A2125
after deploy contract
"skaledDBUsage":{"blocks.db_disk_usage":272286815,"contractStorageUsed":136448,"pieceUsageBytes":89196093,"state.db_disk_usage":23122031}
after fill 1 mb of data 
npx hardhat getStorageUsage --network custom
1048576 bytes
"skaledDBUsage":{"blocks.db_disk_usage":272363541,"contractStorageUsed":1185056,"pieceUsageBytes":89207223,"state.db_disk_usage":22663530}
after selfDestruct
"skaledDBUsage":{"blocks.db_disk_usage":272494557,"contractStorageUsed":1185024,"pieceUsageBytes":89223476,"state.db_disk_usage":22664152}


on 2.1.1
breakable-anguished-tania-borealis
0x7c97049C499A30e204049a99F2281D3F2028e009
deploy
"skaledDBUsage":{"blocks.db_disk_usage":1093688091,"contractStorageUsed":39136,"pieceUsageBytes":15433841,"state.db_disk_usage":143017211}
upload 1 mb
"skaledDBUsage":{"blocks.db_disk_usage":1090131971,"contractStorageUsed":1087744,"pieceUsageBytes":15444995,"state.db_disk_usage":142040055}
1048576 bytes
"skaledDBUsage":{"blocks.db_disk_usage":1090486470,"contractStorageUsed":1087744,"pieceUsageBytes":15488002,"state.db_disk_usage":142040677}

Also, eth_getStorageAt show empty values after invoking self-destruct
3.16.1

> eth.getStorageAt('0xBC8606f37DF53Dce8E0F6bd6277E1d51C9438b56', "0x44faaa7d34c7d33bb0a48136375a3a746e717d5882d9e0a96571c9cd081ef069" )
"0x0000000000000000000000000000000000000000000000000000000000001c11"
> eth.getStorageAt('0xBC8606f37DF53Dce8E0F6bd6277E1d51C9438b56', "0x0" )
"0x0000000000000000000000000000000000000000000000000000000000008000"
> eth.getStorageAt('0xBC8606f37DF53Dce8E0F6bd6277E1d51C9438b56', "0x1" )
"0x00000000000000000000000071cbe3fede33905d4d1bf2bd51f9d4a62375e659"

> eth.getStorageAt('0xBC8606f37DF53Dce8E0F6bd6277E1d51C9438b56', "0x1" )
"0x0000000000000000000000000000000000000000000000000000000000000000"
> eth.getStorageAt('0xBC8606f37DF53Dce8E0F6bd6277E1d51C9438b56', "0x0" )
"0x0000000000000000000000000000000000000000000000000000000000000000"
> eth.getStorageAt('0xBC8606f37DF53Dce8E0F6bd6277E1d51C9438b56', "0x44faaa7d34c7d33bb0a48136375a3a746e717d5882d9e0a96571c9cd081ef069" )
"0x0000000000000000000000000000000000000000000000000000000000000000"

3.17.0-beta.8
Before self-destruct

> eth.getStorageAt('0xC63D41a176b54CCbb421C7Ddf65911b16A975BC7', "0x00a7e657b5a63a54839534ca6890ce88767f226b1483dc57dea10c3336f0548a")
"0x0000000000000000000000000000000000000000000000000000000000000563"
> eth.getStorageAt('0xC63D41a176b54CCbb421C7Ddf65911b16A975BC7', "0x1")
"0x000000000000000000000000b04a7f2cd74eb624df4bd8bed8f7034b5aed89c0"
> eth.getStorageAt('0xC63D41a176b54CCbb421C7Ddf65911b16A975BC7', "0x0")
"0x0000000000000000000000000000000000000000000000000000000000008000"

After self-destruct

> eth.getStorageAt('0xC63D41a176b54CCbb421C7Ddf65911b16A975BC7', "0x0")
"0x0000000000000000000000000000000000000000000000000000000000000000"
> eth.getStorageAt('0xC63D41a176b54CCbb421C7Ddf65911b16A975BC7', "0x1")
"0x0000000000000000000000000000000000000000000000000000000000000000"
> eth.getStorageAt('0xC63D41a176b54CCbb421C7Ddf65911b16A975BC7', "0x00a7e657b5a63a54839534ca6890ce88767f226b1483dc57dea10c3336f0548a")
"0x0000000000000000000000000000000000000000000000000000000000000000"

State_db still have key-value pairs 3.17.0-beta.8 after self-destruct

key:  c63d41a176b54ccbb421c7ddf65911b16a975bc70113557b3410a630f4e95dcd051a3e42b109fe18ede68d73ecbd10ccfef205bd
value:  00000000000000000000000000000000000000000000000000000000000073c3
key:  c63d41a176b54ccbb421c7ddf65911b16a975bc70114dd330fa5b9b7d551f118de8f46fe7ef6f16a5c8fbfb44270e2c48c6a11ba
value:  0000000000000000000000000000000000000000000000000000000000005eeb

[oleksandrSydorenkoJ]

Actual for 3.17.0-develop.60

[kladkogex]

What we find out today that keys are deleted from LevelDB - the calls to delete are executed.

When you delete keys in LevelDB, it does not immediately descrease the size on disk. LevelDB does compaction from time to time, at that point space is freed

There is a way in LevelDB to manually ask to compact a range of keys. It includes lots of code changes, so I am moving this to 2.3.

I

[oleksandrSydorenkoJ]

Verified on 3.17.0-develop.62
Need to re-test on beta

What we find out today that keys are deleted from LevelDB - the calls to delete are executed.

When you delete keys in LevelDB, it does not immediately descrease the size on disk. LevelDB does compaction from time to time, at that point space is freed

There is a way in LevelDB to manually ask to compact a range of keys. It includes lots of code changes, so I am moving this to 2.3.

will be fixed in a separate issue #1682

[oleksandrSydorenkoJ]

Verified on 3.17.0-beta.12

contract a3DfffF1DF4A9F4387b3e1CdCC5cB958dcd84667
1 MB of data cleared in 0.1 second

2023-10-05 13:13:00.477840   createBlock ID = #48481

2023-10-05 13:13:00.477911   Dropping good txn 5a8d4d942d089107348692f9847ee83c9520d17e64bdd5839c9f19691e4338c2

2023-10-05 13:13:00.477947   m_received = 0

2023-10-05 13:13:00.478179   Self-destructing#a3dffff1…
2023-10-05 13:13:00.478197   Iterating over the LevelDB prefix: �����J�C�����\�X��Fg
2023-10-05 13:13:00.589864   Self-destruct cleared values:32770

2023-10-05 13:13:00.709771   Block sealed #48481 (#ac687eca…)
2023-10-05 13:13:00.709808   Block stats:BN:48480:BTS:1696511577:TXS:3:HDRS:11:LOGS:40:SENGS:1:TXRS:61:BLCKS:3:ACCS:150:BQS:1:BDS:14552:TSS:0:UTX:0:VTX:0:CMM:15009185:KDS:32779

state DB:
before self-destruct:

Keys starting from LevelDB prefix a3DfffF1DF4A9F4387b3e1CdCC5cB958dcd84667: 32772
total key amount:  131441
total value amount: 131441

After seldFesdtruct function

Keys starting from LevelDB prefix a3DfffF1DF4A9F4387b3e1CdCC5cB958dcd84667: 0
Total key amount:  98669
Total value amount: 98669