Sharding schemes over hpaxos

[cwgoes]

It would be helpful to be able to instantiate a sort of flexible sharding utilising heterogenous Paxos, where a validator set controlled by a single logical proof-of-stake system and staking token can be split across "shards" - independent consensus instances which execute in parallel, but where the validator subsets can be guaranteed to have certain amounts of overlap by the assignment logic of the logically centralised proof-of-stake module.

Ideally we would be able to craft an automatic scheduling algorithm so that transactions can use a unified programming model ("as-if-atomic") and the scheduling mechanism can produce an actual execution which is serialisable per a particular ordering but splits actual execution amongst shards efficiently (as efficiently as possible). This intersects nontrivially with Ferveo, as we want to commit to an ordering before knowing transaction contents (making scheduling considerably more difficult), but it should still be possible to run the scheduling algorithm after we know the order and transactions are decrypted - this just fixes a particular order (of serialisability) which actual execution must be semantically equivalent to.

Prior research:

• https://arxiv.org/pdf/2107.07297.pdf

[junkicide]

Condensed summary of Shard Scheduler operation: https://hackmd.io/vtNbbrgMToS0elY7oX5XXQ