Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise updating events, use Entity.create #63

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

stwiname
Copy link
Contributor

@stwiname stwiname commented Nov 27, 2024

When blocks have large numbers of extrinsics and events the events would get iterated many times creating really poor performance.

E.g. This block goes from taking ~11m to ~1.5s https://kusama.subscan.io/block/25845989

It also updates creating entities from new Entity to Entity.create

Upgrade instructions

This creates project upgrades for all dictionaries on the network. Polkadot and Kusama have upgrades at the start of the large blocks from last week and the rest have upgrades at blocks at the time of creating.

DO $$
DECLARE  
   -- Update the schema to match the project upgrade height
   height numeric := 25845989;  
BEGIN  
	-- Update the schema to match the project
	SET search_path TO kusama;
        delete from events where block_height > height;
	delete from extrinsics where block_height > height;
	delete from spec_versions where block_height > height;

        -- Uncomment for networks with EVM
	delete from evm_transactions where block_height > height;
	delete from evm_logs where block_height > height;

	-- Uncomment for networks with WASM Contracts
	delete from contracts_calls where block_height > height;
	delete from contracts_emitted where block_height > height;


	update "_metadata"  set value = to_jsonb(height) where key = 'lastProcessedHeight';
	delete from _poi where id > height;
        reset search_path;
END $$;

@stwiname stwiname force-pushed the optimise-iterate-events branch from 781d1c0 to 74d0558 Compare November 27, 2024 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant