Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor property writing + dev mode enhancements #1000

Merged
merged 27 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
81bcea4
fix: set token props only if something has changed
mrshiposha Sep 25, 2023
e3c9079
fix: dont read storage during minting when not needed
mrshiposha Sep 25, 2023
9c8e078
feat: add reset_token_properties benchmark
mrshiposha Sep 25, 2023
d63678e
feat: add integration-tests profile and use in CI
mrshiposha Sep 25, 2023
5ec8b82
chore: add temporary nft/rft weights
mrshiposha Sep 25, 2023
331f0fa
fix: cache chain build in CI
mrshiposha Sep 26, 2023
192f49d
fix: dont read tpp when not needed
mrshiposha Sep 26, 2023
9b4620c
chore: repeat temporary nft/rft benches
mrshiposha Sep 26, 2023
12a441c
fix: debug check if new token doesnt have any properties
mrshiposha Sep 26, 2023
915db56
chore: bench weights
mrshiposha Sep 26, 2023
cb1a9d8
fix: calibrate weights
mrshiposha Sep 26, 2023
7cc9052
fix: bench profile production
mrshiposha Sep 26, 2023
1acfc39
chore: bench weights repeats 400
mrshiposha Sep 27, 2023
d2c9363
tests: performance test for NFT tokens
uandysmith Sep 27, 2023
44071b6
refactor: use type-safe propertywriter to set/delete properties
mrshiposha Sep 30, 2023
2657b80
fix: EREFUSED due to ws://localhost (?)
mrshiposha Sep 30, 2023
0900b3d
chore: temporary bench nft/rft/common
mrshiposha Sep 30, 2023
42bd0c6
chore: bench nft/rft/common
mrshiposha Sep 30, 2023
74a28c8
chore: calibrate weight2fee
mrshiposha Sep 30, 2023
b6b1e0f
feat: add cli param disable-autoseal-on-tx
mrshiposha Oct 2, 2023
184b30a
feat: finalization can be enabled in dev mode
mrshiposha Oct 2, 2023
57b7f8c
fix: slightly fix broken pov-estimate
mrshiposha Oct 2, 2023
e105d72
fix: use config.ts for endpoints
mrshiposha Oct 2, 2023
5ef5e6b
fix: make perf test seq and remove unused functions
mrshiposha Oct 2, 2023
eebd0e2
fix: use OptionQuery for TokenProperties
mrshiposha Oct 2, 2023
6b27551
fix: unit tests
mrshiposha Oct 2, 2023
630fc89
fix: use lowercase letters in EVM errors
mrshiposha Oct 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .docker/Dockerfile-chain-dev
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ COPY . /dev_chain

WORKDIR /dev_chain

RUN cargo build --release
RUN cargo build --profile integration-tests --features=${NETWORK}-runtime
RUN echo "$NETWORK"

CMD cargo run --release --features=${NETWORK}-runtime -- --dev -linfo --rpc-cors=all --unsafe-rpc-external
CMD cargo run --profile integration-tests --features=${NETWORK}-runtime -- --dev -linfo --rpc-cors=all --unsafe-rpc-external
CertainLach marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion .docker/Dockerfile-unique
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ RUN --mount=type=cache,target=/cargo-home/registry \
--mount=type=cache,target=/unique_parachain/unique-chain/target \
cd unique-chain && \
echo "Using runtime features '$RUNTIME_FEATURES'" && \
CARGO_INCREMENTAL=0 cargo build --release --features="$RUNTIME_FEATURES" --locked && \
CARGO_INCREMENTAL=0 cargo build --profile integration-tests --features="$RUNTIME_FEATURES" --locked && \
mv ./target/release/unique-collator /unique_parachain/unique-chain/ && \
cd target/release/wbuild && find . -name "*.wasm" -exec sh -c 'mkdir -p "../../../wasm/$(dirname {})"; cp {} "../../../wasm/{}"' \;

Expand Down
2 changes: 1 addition & 1 deletion .docker/docker-compose.gov.j2
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ services:
options:
max-size: "1m"
max-file: "3"
command: cargo run --release --features={{ NETWORK }}-runtime,gov-test-timings -- --dev -linfo --rpc-cors=all --unsafe-rpc-external
command: cargo run --profile integration-tests --features={{ NETWORK }}-runtime,gov-test-timings -- --dev -linfo --rpc-cors=all --unsafe-rpc-external
4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ inherits = "release"
lto = true
opt-level = 3

[profile.integration-tests]
inherits = "release"
debug-assertions = true

[workspace.dependencies]
# Unique
app-promotion-rpc = { path = "primitives/app_promotion_rpc", default-features = false }
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ evm_stubs: UniqueFungible UniqueNFT UniqueRefungible UniqueRefungibleToken Contr

.PHONY: _bench
_bench:
cargo run --release --features runtime-benchmarks,$(RUNTIME) -- \
cargo run --profile production --features runtime-benchmarks,$(RUNTIME) -- \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Я подумал, а зачем мы вообще это кучу раз компилируем...
У нас там перед каждым бенчмарком палеты ноду снова пересобирает, и я только что понял что она пересобирает - у нас изменяется weights.rs, и оно собирает ноду с новыми весами.

По идее тут нужно один раз ноду собрать, и потом вызывать ./unique-collator benchmark ...

Что тут однако не выйдет сделать, поскольку у нас рекурсивный make.
Сейчас unactionable, но способ оптимизации ясен.

benchmark pallet --pallet pallet-$(if $(PALLET),$(PALLET),$(error Must set PALLET)) \
--wasm-execution compiled --extrinsic '*' \
$(if $(TEMPLATE),$(TEMPLATE),--template=.maintain/frame-weight-template.hbs) --steps=50 --repeat=80 --heap-pages=4096 \
Expand Down
12 changes: 11 additions & 1 deletion node/cli/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,20 @@ pub struct Cli {
/// an empty block will be sealed automatically
/// after the `--idle-autoseal-interval` milliseconds.
///
/// The default interval is 500 milliseconds
/// The default interval is 500 milliseconds.
#[structopt(default_value = "500", long)]
pub idle_autoseal_interval: u64,

/// Disable auto-sealing blocks on new transactions in the `--dev` mode.
#[structopt(long)]
pub disable_autoseal_on_tx: bool,

/// Finalization delay (in seconds) of auto-sealed blocks in the `--dev` mode.
///
/// Disabled by default.
#[structopt(long)]
pub autoseal_finalization_delay: Option<u64>,

/// Disable automatic hardware benchmarks.
///
/// By default these benchmarks are automatically ran at startup and measure
Expand Down
5 changes: 1 addition & 4 deletions node/cli/src/command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ use sc_cli::{
use sc_service::config::{BasePath, PrometheusConfig};
use sp_core::hexdisplay::HexDisplay;
use sp_runtime::traits::{AccountIdConversion, Block as BlockT};
use std::{time::Duration};

use up_common::types::opaque::{Block, RuntimeId};

Expand Down Expand Up @@ -481,14 +480,12 @@ pub fn run() -> Result<()> {
if is_dev_service {
info!("Running Dev service");

let autoseal_interval = Duration::from_millis(cli.idle_autoseal_interval);

let mut config = config;

config.state_pruning = Some(sc_service::PruningMode::ArchiveAll);

return start_node_using_chain_runtime! {
start_dev_node(config, autoseal_interval).map_err(Into::into)
start_dev_node(config, cli.idle_autoseal_interval, cli.autoseal_finalization_delay, cli.disable_autoseal_on_tx).map_err(Into::into)
};
};

Expand Down
33 changes: 27 additions & 6 deletions node/cli/src/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -169,9 +169,9 @@ pub struct AutosealInterval {
}

impl AutosealInterval {
pub fn new(config: &Configuration, interval: Duration) -> Self {
pub fn new(config: &Configuration, interval: u64) -> Self {
let _tokio_runtime = config.tokio_handle.enter();
let interval = tokio::time::interval(interval);
let interval = tokio::time::interval(Duration::from_millis(interval));
CertainLach marked this conversation as resolved.
Show resolved Hide resolved

Self { interval }
}
Expand Down Expand Up @@ -885,7 +885,9 @@ pub struct OtherPartial {
/// the parachain inherent
pub fn start_dev_node<Runtime, RuntimeApi, ExecutorDispatch>(
config: Configuration,
autoseal_interval: Duration,
autoseal_interval: u64,
autoseal_finalize_delay: Option<u64>,
disable_autoseal_on_tx: bool,
) -> sc_service::error::Result<TaskManager>
where
Runtime: RuntimeInstance + Send + Sync + 'static,
Expand All @@ -912,7 +914,10 @@ where
+ sp_consensus_aura::AuraApi<Block, AuraId>,
ExecutorDispatch: NativeExecutionDispatch + 'static,
{
use sc_consensus_manual_seal::{run_manual_seal, EngineCommand, ManualSealParams};
use sc_consensus_manual_seal::{
run_manual_seal, run_delayed_finalize, EngineCommand, ManualSealParams,
DelayedFinalizeParams,
};
use fc_consensus::FrontierBlockImport;

let sc_service::PartialComponents {
Expand Down Expand Up @@ -980,20 +985,22 @@ where
.pool()
.validated_pool()
.import_notification_stream()
.filter(move |_| futures::future::ready(!disable_autoseal_on_tx))
.map(|_| EngineCommand::SealNewBlock {
create_empty: true,
finalize: false, // todo:collator finalize true
finalize: false,
parent_hash: None,
sender: None,
}),
);

let autoseal_interval = Box::pin(AutosealInterval::new(&config, autoseal_interval));

let idle_commands_stream: Box<
dyn Stream<Item = EngineCommand<Hash>> + Send + Sync + Unpin,
> = Box::new(autoseal_interval.map(|_| EngineCommand::SealNewBlock {
create_empty: true,
finalize: false, // todo:collator finalize true
finalize: false,
parent_hash: None,
sender: None,
}));
Expand All @@ -1003,6 +1010,20 @@ where
let slot_duration = cumulus_client_consensus_aura::slot_duration(&*client)?;
let client_set_aside_for_cidp = client.clone();

if let Some(delay_sec) = autoseal_finalize_delay {
let spawn_handle = task_manager.spawn_handle();

task_manager.spawn_essential_handle().spawn_blocking(
"finalization_task",
Some("block-authoring"),
run_delayed_finalize(DelayedFinalizeParams {
client: client.clone(),
delay_sec,
spawn_handle,
}),
);
}

task_manager.spawn_essential_handle().spawn_blocking(
"authorship_task",
Some("block-authoring"),
Expand Down
Loading
Loading