Improve Waku resilience against flooding attacks

[jm-clius]

Background

During the recent Status client offsite, Waku's vulnerability to flooding-type DoS attacks were highlighted to be of particular concern for Status Communities. Since, in general, a pubsub topic is not rate limited and all messages are routed without validation, anyone can potentially join the network and spam/flood the network pubsub topic with random data until parts or all of the network is down.

Problem statement

Three main attack vectors (from Community's perspective) were identified:
a. attack the entire Waku Network (flooding the pubsub topic) and potentially take down multiple communities
b. attack a community (flooding the content topic) - even if this community-level attack is successful, this should not affect other communities
c. flood a community with join requests

From my perspective, protecting the Waku Network (i.e. pubsub topic) against high-rate attacks forms the basis of addressing all three attack vectors identified above. If someone can flood the pubsub topic, there is little value in rate-limiting one isolated content topic, as all nodes will be routing all messages in any case.

Solution considerations

1. Waku's main proposition for rate limited relay is RLN relay. This may not be feasible to implement by Status Communities (especially for network-wide rate control) within the short/medium term.
2. Application-level validation is likely to form part of the solution. This is conceptually similar to how many other open p2p networks fight against spam (by validating messages that are "allowed" to be routed).
3. Application-level validation could include some membership validation (perhaps involving a decentralised CAPTCHA?) or e.g. more specific community membership. Note, however, that for network wide protection, a validation rule should apply to all applications/content topics allowed on that pubsub topic.
4. Well-designed pubsub topic sharding is likely to form part of the solution. This both divides the network between "spammable" domains and allow different validation rules for different shards.
5. Sharding should be a well-considered as it implies possibly significant tradeoffs in anonymity and privacy - receiver anonymity, for example, relies on the k-anonymity of multiple content topics within a single network.

cc @staheri14 @cammellos @richard-ramos

[fryorcraken]

Maybe worth mention that (2) is pursued with #141. While it is initially meant to save some space, the same validation could be apply at the gossip sub layer to not forward invalid message and downgrade peer score of peers that do (not all peer may apply the validation hence a given peer may prefer to connect to peers that do apply the validation).

[oskarth]

I think we have to be realistic about there not being a free lunch here, in terms of having a permissionless and sybil-resistant p2p system, what are Status users willing to do?

For example, RLN-Relay is the main way we've been approaching this. It currently requires a financial stake, which I believe is a no go for the Status app. Would having a ZK social web2 based registration mechanism (RLN+Interrep) be acceptable to form a group? #147 - this might require some form of relayer to update smart contract to ensure users can register with no funds and only with social reputation.

Application-specific validation is useful and can make sense for communities, but there are some gotchas here, e.g. (a) wrt to proving you belong to a community and how this works in terms of decrypt/encrypt (b) how this deals with private messages / group messages outside of a given community.

Topic sharding is also useful, and helps mitigate things a bit, but it doesn't address the problem in a fundamental way for a motivated attacker (unlike say RLN).

Basically, what are we willing to give up and what are the trade-offs we are willing to make to achieve this goal?

I also want to point out that it is quite late for fundamental changes to the design, so I'd look at whatever we introduce as ways of "hardening" against attacks like this, and not make this a blocker for general integration and deployment. For example, this is how the RLN Relay mechanism has been designed, where it something that can be opted in for communication channels where users put a premium on censorship resistance. In this case, the trade-off is quite clear (economic registration in exchange for economic security guarantees on message rate).

[rymnc]

Adding to the previous comment, having application specific validation would enable censorship, so I am not sure if that is the way forward for spam prevention.

[Menduist]

Application-level validation is likely to form part of the solution. This is conceptually similar to how many other open p2p networks fight against spam (by validating messages that are "allowed" to be routed).

Note that this only works if a high percentage of the waku network (>90%) applies the rules.

GossipSub is design to work (ie propagate a message to the whole network) in adversarial environment with many sybils. And you can consider nodes with validation as a "sybil" (since they will swallow some messages that nodes without validation will consider valid), so you need a lot of sybils to actually stop the propagation.

I don't see how that could work in a generic network like Waku. Maybe if you have some way to set rules for content topic globally, and everyone follows the rules? That would require some own-chain owner of a content topic that can set rules for that topic, and require every waku node to follow chain (or maybe messages could include a light proof that they are allowed on this topic somehow). And not sure what the rules would look like, seems like we would be back to RLN (that would incur fees everytime you add / remove someone again)
Off chain would also be possible, but then the owner would need to broadcast the rules frequently enough that ~everyone in the network has them

Otherwise, the only benefit will be to avoid storing invalid messages, which will at least save some space on disk, but will have little impact on bandwidth

the same validation could be apply at the gossip sub layer to not forward invalid message and downgrade peer score of peers that do

How to split your network in 2, 101 :)

[rymnc]

RLN-Interep context: #147 (comment)

In the context of status communities, each community could have their own merkle tree and set of relayers (or maybe even one relayer) which adds the updated root to the Interep contract. It does require a certain degree of trust, and I am not sure if communities are designed that way.

[Menduist]

I have proposal for this: https://hackmd.io/vSpt3sJ-RcOYjb6jLmkWDg

It's based on off-chain RLN like I talked about in my last comment, and seems like a good trade-off given the fundamental impossibility of privacy-protecting spam protection without Proof-Of-Money / PoW

[rymnc]

I have proposal for this: https://hackmd.io/vSpt3sJ-RcOYjb6jLmkWDg

It's based on off-chain RLN like I talked about in my last comment, and seems like a good trade-off given the fundamental impossibility of privacy-protecting spam protection without Proof-Of-Money / PoW

Very interesting proposal! However, I do think one of the requirements for communities is zero barrier of entry for the community owner too (I may be wrong here).

[Menduist]

For community owner, they could either use the "free mode" (which may loose messages when the network is under heavy load), or go to a broker to get some dedicated bandwidth according to the broker terms (which may or may not include money), or directly buy bandwidth on-chain (which would probably only be used by big communities as this would be expensive)

While this is not ideal, I think it's the best we can do

[alrevuelta]

@Menduist Really like your proposal. Some comments + ideas on top.

Regarding your bandwidth nft, I assume you mean a sould bound NFT? Otherwise, you can send it to someone else and since you suggest that you don't need to follow the chain, you can end up having sybils. Or how do you keep track of people transferring it without following the chain?

Also not sure I understand the merkleRoot and proof fields. Is the proof a merkle proof, or some other kind of proof? Can you generate this proof without the Ethereum state? What are the leaves of the merkle tree below the merkleRoot? Where are they taken from?

My initial naive approach when I read your proposal was. Let GAB be a guaranteed available bandwidth token with a total supply of 1000, where each one gives you the right of publishing 1mb/hour. All tokens are minted in Ethereum and are owned by different addresses A_n. One simple proof would be to include a signature of the message as a field, so if it was signed by an account that owns a GAB then we will propagate that message if the rate has not been exceeded.

Some cons:

• Checking this A to GAB mapping requires following the Ethereum blockchain, which requires running a light node which perhaps would be too much for desktop users.
• Your Ethereum key would need to be "hot" in your node, perhaps also a no go. I'm sure there are ways of overcoming this.
• Now we can map message/eth-address, sacrificing some privacy. Can we deal with this?

Adding to the previous comment, having application-specific validation would enable censorship, so I am not sure if that is the way forward for spam prevention.

As long as we have the "free-mode" that @Menduist suggests, I think it's a good tradeoff. In this case, it would be rate-limiting more than censorship. On top of this, if our GAB tokens support Ethereum well-known standards, they can be sold in secondary markets purely based on supply(fixed)/demand. So it's not just tokens that we create and we control, but tokens that can be traded for a share in our network's bandwidth in a decentralized manner. It's a way of putting a price to a scarce resource. This is a bit different than the RLN (staking + slashing) approach, since we don't slash, just rate limit until some time has passed. Waku bandwidth is the new Ethereum blockspace.

My knowledge in interep is quite limited, and guess its zk nature is a huge advantage, but by using Ethereum's credentials and ecosystem in general there are lots of stuff we can leverage.

TLDR:

• Piggyback on ethereum, nice but suitable for desktop users?
• Off-chain rate limiting based on holding a limited supply +1 (@Menduist idea)
• Rules encoded in a smart contract so that we can rate limit while not being able to censor.

Ofc all these ideas are related to short-term scaling with DoS protection. Wondering if we should move the conversations here #164.

[Menduist]

Otherwise, you can send it to someone else and since you suggest that you don't need to follow the chain, you can end up having sybils

No need to be soul-bound, let's say there are 1000 nfts. Everytime one is updated, you simply remove the previous parameters associated with this NFT, which naturally protects against double spend. Example:

• Alice: holds nft n43. Send NftParam to let her community use the associated bandwidth
• Alice: transfers nft 43 to Bob
• Bob: Send NftParam to let his community use it
• Entire network: deletes the old parameters associated with NFT n43, so alice community can't use it anymore

That is the nice thing about the NFT. This is detailed in the "NFT vs ERC20" section

Also not sure I understand the merkleRoot and proof fields

proof is an ethereum light proof. It's a merkle proof that you hold the NFT at a specific block, and a second proof that proves that this block is valid. You don't need to follow chain to verify it, but you do need access to chain to generate them (using eth_getProof & others)
So users holding NFTs wanting to change their NftParam will need chain access (infura, whatever, since will be proved anyway)

merkleRoot is the RLN root, which basically sets the list of users who are allowed to use the GAB of this NFT

My initial naive approach when I read your proposal was.

Yes, the added complexity of my proposal compared to yours is to alleviate the cons you listed.

[alrevuelta]

@Menduist I see, but we need an Ethereum node. So every node participating in gossipsub would need a synced node. Perhaps a light one is enough, but not sure if this is something we can assume.

That is the nice thing about the NFT. This is detailed in the "NFT vs ERC20" section

"it would be more fined tuned & fluid than NFTs". Why? I mean the use that we are giving nfts here is not really non-fungible. My 1mb/h and your 1mb/h is the same. And If I get yours 1+1=2mb/h so perhaps it makes more sense to go with an ERC20 approach. Better also in terms of liquidity if at some point they end up being traded in an AMM, but that another topic.

[Menduist]

I see, but we need an Ethereum node

Just to clarify, this kind of light proof just requires each node to download 25kb/day to "follow chain" and be able to verify proofs. The nodes could pull this from any ethereum public provider (since that will be verified anyway), or just bundle the required data in the NftParams messages.
As said in the intro of this proposal, the goal is that everyone can apply these rules.

the use that we are giving nfts here is not really non-fungible.

It is as far a "double spend" is concerned. Your NFT allows you to set your parameters, not mine.
ERC20 does seem nicer, since we don't have to set some arbitrary "1 nft = 1 mb/h", where it's impossible to get <1mb/h, and painful to get >10mb/h (since you have to buy a lot of NFTs)

But ERC20 requires extra protection against double spend. If we think "at least one node on the network is following full chain, and extra complexity" is worth it, that's doable