Ability to configure several RPC (failover) urls

[akolotov]

At this moment it is possible to configure on one RPC URL for each bridge direction:

HOME_RPC_URL=https://kovan.infura.io/mew
FOREIGN_RPC_URL=https://kovan.infura.io/mew

It means that if the remote node will not be accessible (e.g. the link is down or the node was shut down) the bridge will not be able to get events and send transactions by using the corresponding RPC channel.

In order to increase the availability of the bridge operations it is suggested to introduce the ability to configure a list of RPC URLs:

HOME_RPC_URL="https://sokol.poa.network/, https://some.rpc.node.at.home/"
FOREIGN_RPC_URL=https://kovan.infura.io/mew, https://some.rpc.node.at.foreign/

In this case, the bridge should switch to the next RPC link in the list if the previous URL in the list is not accessible. As soon as the link is switched the corresponding log message should appear, e.g.

The RPC channel (<url is here>) is not accessible. Switching to the next RPC link (<url is here>)

If only one RPC link is configured the bridge should use a random delay (e.g. from 5 to 30 seconds) to try to reconnect to the RPC link.

If the end of the list is reached the first URL from the list must be checked again and the bridge should switch to this channel.

If all channels are not available the bridge should use a random delay (e.g. from 5 to 30 seconds) to try to reconnect to the first RPC link from the list.

[fvictorio]

We are using the RPC URLs in two ways: as HTTP endpoints (for sending raw txs) and for creating web3 instances.

The HTTP endpoints part is not hard: we should have a function that receives a list of URLs and a callback, tries the callback in each URL and either returns the result or retries. (There are several details to be taken into account here, but they are not that important.)

The web3 part is what concerns me. I can think of several options here:

1. Create a web3 instance for each URL and cycle through them until one works. The problem here is that detecting if a failure was caused by the endpoint or by other reason seems a little tricky. The other issue is that we are also creating contract instances: we should make a contract instance for each web3 instance... It doesn't seem too elegant.
2. Create a custom web3 provider that receives several URLs and cycles through them. That way we would create just one web3 instance and the provider would handle the retries. Of course, doing this is harder, but it could be a nice by-product, useful beyond the bridge. I investigated a little and it doesn't seem that hard to do, but, of course, it's not without risk.
3. Don't use web3 and handle everything through raw HTTP calls. This could work, and it would use the same function that we'd make for sending transactions, but it would mean doing a non-trivial refactor.

I think the 2nd option is a good one, but it would take some time to develop and test. If that's not a possibility, I guess the 1st option could work.

[akolotov]

Thanks @fvictorio for the summary.
Did you investigate why sending raw transactions through HTTP is more preferable than by using Web3 Provider?
I see that the 2nd option is most logical especially if there is no limitation in sending raw transactions by Web3 Provider as well. I think we could even suggest this feature for inclusion into JS implementation of Web3 API.

[fvictorio]

We are sending raw HTTP for sending the transactions, since that way it's easier to send multiple transactions with consecutive nonces.

I will work on the provider approach then, thanks!

[akolotov]

Is it possible to use sendSignedTransactiion which also allows to manipulate with NOnces?

https://web3js.readthedocs.io/en/1.0/web3-eth.html#sendsignedtransaction

[fvictorio]

I already have something working that does the fallback both when using HTTP and with web3 (they are separate implementations). There are some pending things to do, but after they are done I will create the PR with that.

But doing everything through web3 sounds cleaner and easier to maintain, so let's keep that option open and maybe we can do refactor so that everything goes through web3 (that can be done in that PR or in a future one).

[fvictorio]

But doing everything through web3 sounds cleaner and easier to maintain, so let's keep that option open and maybe we can do refactor so that everything goes through web3 (that can be done in that PR or in a future one).

@akolotov I forgot to follow up on this: we can't use web3 for raw transactions because the promise resolves when the transaction is mined, while we just wait for the transaction to be received.

Maybe we could do something around this: wait until all txs are mined (but this would mean that the time to mark the job as finished would increase a lot) or check if we can use the event emitter of the response (instead of the promise) to finish when the transaction is received.

[akolotov]

@fvictorio let's continue the discussion in #61. I would prefer the second approach - without usage of the promise.