Explorer/API bugs and general improvement suggestions

[chainum]

Failing requests

It seems that the explorer is occasionally struggling to pull data from the ElasticSearch endpoint/server (https://elastic-aws.elrond.com) and that the requests sent to:

• https://elastic-aws.elrond.com/tps/_doc/meta
• https://elastic-aws.elrond.com/transactions/_search
• https://elastic-aws.elrond.com/transactions/_count

fail/timeout after 3 seconds (as defined in /static/js/context/state.tsx).

This leads to that the UI sometimes initially displays address info when looking up an address (since this data is from https://wallet-api.elrond.com/address/ADDRESS) but then switches over to not displaying anything:

This usually coincides with an influx of network traffic so I'm assuming that the ElasticSearch server(s) isn't coping all too well as soon as traffic/txs starts to increase.

Not sure how much everything would improve by just increasing the timeout (e.g. from 3 to 6 seconds) in the React app, so that the ElasticSearch API requests have a higher chance of succeeding, but could be worth looking into.

It might also be a good idea to at least let users partially see the address balance (since it's not served via the ElasticSearch API) and just display "transactions aren't available right now" for the actual transaction list that can't be loaded.

New wallets without any transactions:

Also, when looking up non-existent addresses (i.e. addresses that are yet to receive their first transaction), the Explorer UI will look exactly the same as explained above regarding the failed API requests:

I suggest that you to update the UI to function more like it does on Etherscan:

I.e: Even if a wallet technically doesn't exist yet - it's better UI wise to simply display that there are no transactions vs. "Unable to locate this address hash" like the Explorer currently does.

Use WebSockets instead of REST API polling

The three requests mentioned above will also get fired off every 3 seconds so if you just leave a browser window open you'll generate a bunch of transactions to the ElasticSearch API:

Ideally this data should be transmitted using WebSockets instead of polling the data every 3 seconds over HTTPS.

One potential solution could be to run one or several websocket servers accessible on e.g. https://elastic-ws.elrond.com that routinely pulls the data from the underlying ElasticSearch server(s) (https://elastic-aws.elrond.com) and only propagates actual updates to the connected javascript/React WebSocket consumer client on https://explorer.elrond.com.

It also seems that the UI would greatly benefit from implementing WebSockets for other features as well - for example auto-updating the balance of an address when looking it up in the explorer. Currently only transactions are auto-updated - but not the address balance. This results in an inconsistent UI.

Deploy multiple API endpoints

I personally think that no one should be required to run a validator or observer node to send or receive tokens. Right now all API requests for the main API are routed via https://wallet-api.elrond.com which I assume then proxy the requests to the correct underlying observers.

When I performed my attacks (#799 / #829) the spamming nuked the wallet API relatively fast as soon as I switched over to using that API as the endpoint for my transactions.

A more resilient way to manage increased load to the official / wallet API could be to split up the api into several separate subdomains split per shard and then put e.g. Elastic Load Balancing in front that load balances to a bunch of shard specific observers for a given shard. E.g:

• https://api.s0.m.elrond.com - shard 0 API endpoint for Mainnet
• https://api.s1.m.elrond.com - shard 1 API endpoint for Mainnet
• https://api.s2.m.elrond.com - shard 2 API endpoint for Mainnet
• https://api.s3.m.elrond.com - shard 3 API endpoint for Mainnet
• https://api.s4.m.elrond.com - shard 4 API endpoint for Mainnet
• https://api.meta.m.elrond.com - meta shard API endpoint for Mainnet

(.m. can obviously just be replaced with .t. for a testnet setup etc.)

Then clients just implement the logic for selecting the appropriate endpoint using for example something like this:

// APIEndpointForAddress - identifies what endpoint to use for a specific address
func APIEndpointForAddress(address string, networkType string) string {
	lastAddressCharacter := strings.ToLower(string(address[len(address)-1]))

	var shard int

	switch lastAddressCharacter {
	case "0", "8":
		shard = 0
	case "1", "5", "9", "d":
		shard = 1
	case "2", "6", "a", "e":
		shard = 2
	case "3", "7", "b", "f":
		shard = 3
	case "4", "c":
		shard = 4
	}

	return fmt.Sprintf("https://api.s%d.%s.elrond.com", shard, networkType)
}

This way the official API can still scale sufficiently to deal with a multitude of clients not propagating txs via their own set of nodes.

This also makes it super easy for people to get started with clients/apps/API interactions (iOS, Android apps etc. etc.) without having to bother with running a validator or an observer node in order to interact with Elrond's blockchain.

[iulianpascalau]

@LucianMincu