Calculate VID dispersal locally

[lukaszrzasik]

Closes #2731

This PR:

Splits DAProposalRecv into DAProposalRecv and DAProposalValidated.
DA task handles DAProposalRecv by validating DA proposal and emitting DAProposalValidated if correct.
DA task handles DAProposalValidated by creating, signing and sending the vote.
VID task handles DAProposalValidated by calculating VID dispersal and emitting VidDisperseRecv for each VID share.
The newly calculated VID dispersal is stored locally.
Modifies VID disperse validity check: VID is valid if signed by the leader or by the node itself meaning this is a locally generated VID.

This PR does not:

Key places to review:

crates/task-impls/src/consensus.rs
crates/task-impls/src/da.rs
crates/task-impls/src/vid.rs

How to test this PR:

The change was tested by applying this patch:

diff --git a/crates/task-impls/src/network.rs b/crates/task-impls/src/network.rs
index ade3b12..4879a81 100644
--- a/crates/task-impls/src/network.rs
+++ b/crates/task-impls/src/network.rs
@@ -285,7 +285,8 @@ impl<
                 )
             }
             HotShotEvent::VidDisperseSend(proposal, sender) => {
-                return self.handle_vid_disperse_proposal(proposal, &sender);
+                return None;
+                // return self.handle_vid_disperse_proposal(proposal, &sender);
             }
             HotShotEvent::DAProposalSend(proposal, sender) => {
                 maybe_action = Some(HotShotAction::DAPropose);

and checking if the tests still pass. The patch disables sending the VID dispersal by the leader so that the DA members can only use the locally generated VID dispersal.

Additionally tests in:
crates/testing/tests/da_task.rs and
crates/testing/tests/vid_task.rs
has been modified to test the modified approach.

[lukaszrzasik]

I'm thinking about splitting DAProposalRecv into two parts:

1. Proposal validation
2. Creating, signing and sending the vote.

A new HotShotEvent called DAProposalValidated would be emitted after proposal validation.
DA Task can handle it by continuing with the vote and VID task in parallel can calculate the VID dispersal.

Does that sound like a good idea?

[bfish713]

I'm thinking about splitting DAProposalRecv into two parts:

1. Proposal validation
2. Creating, signing and sending the vote.

A new HotShotEvent called DAProposalValidated would be emitted after proposal validation. DA Task can handle it by continuing with the vote and VID task in parallel can calculate the VID dispersal.

Does that sound like a good idea?

Yes this sounds like a good idea to avoid duplicate validation, and to avoid waiting for VID calculation to vote

[bfish713]

Changes look good to me, just one spot we might want to simplify.

We still need to omit sending VID over the backup network when using the combined network. But that's can be a seperate PR

[bfish713]

why do we wrap the mapped function in Option here?

[lukaszrzasik]

My idea here was: to_proposal can return None when signing the proposal failed. When it happens the closure returns None, if it succeeds it returns Some(_). Then we check if all the proposals have been successfully signed and only if it's true we store all the VID dispersal shares.

So the logic is: if only one signing fails, we don't store the VID shares.

I'm not sure if this logic is correct but it seemed correct to me.
What do you think?

[bfish713]

Oh I missed the ? operator. I think it's fine, but would prefer if all was filter. I think if any fail to sign just the ones that weren't signed would be excluded.

[lukaszrzasik]

No problem. Changed.