Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add exponential backoff for withdrawals queries #2133

Merged
merged 33 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
4588f8c
don't fetch withdrawals if nonce is zero
spsjvc Dec 9, 2024
fb8e92f
fix comments
spsjvc Dec 9, 2024
57d4a29
rename l2Provider to provider
spsjvc Dec 9, 2024
b153022
wip
spsjvc Dec 9, 2024
565ad23
reduce diff
spsjvc Dec 9, 2024
edcd366
remove duplicate check
spsjvc Dec 9, 2024
5992894
oops
spsjvc Dec 9, 2024
fc97519
Merge branch 'master' into perf-sequential-tx-history
spsjvc Dec 9, 2024
e9c8f56
dupe check
spsjvc Dec 9, 2024
b158c14
add backOff
spsjvc Dec 9, 2024
a33a55f
clean up
spsjvc Dec 9, 2024
92e66ec
clean up
spsjvc Dec 10, 2024
924e0a4
use custom custom gateways
spsjvc Dec 10, 2024
e15683a
clean up
spsjvc Dec 10, 2024
75d9823
clean up
spsjvc Dec 10, 2024
3941f7b
clean up more
spsjvc Dec 10, 2024
6067f9c
clean up
spsjvc Dec 10, 2024
43a9f82
clean up
spsjvc Dec 10, 2024
aeaa9a9
clean up more
spsjvc Dec 10, 2024
33b2240
revert exponential back off for now
spsjvc Dec 10, 2024
b45452b
wrap up
spsjvc Dec 10, 2024
57ef6b9
increase delay
spsjvc Dec 10, 2024
7bcda1f
Merge branch 'master' into perf-sequential-tx-history
spsjvc Dec 10, 2024
82c13c1
use awaited
spsjvc Dec 10, 2024
eedf57a
install exponential-backoff
spsjvc Dec 10, 2024
058674e
add backoff
spsjvc Dec 10, 2024
5baa40b
move
spsjvc Dec 10, 2024
751be46
move file
spsjvc Dec 10, 2024
8ec6b17
fetch eth and token withdrawals sequentially
spsjvc Dec 10, 2024
5737c31
Merge branch 'perf-sequential-tx-history' into feat-exponential-backoff
spsjvc Dec 10, 2024
a68c4f9
yup
spsjvc Dec 10, 2024
b90e4d9
Merge branch 'master' into feat-exponential-backoff
spsjvc Dec 12, 2024
65ff6e7
weird
spsjvc Dec 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/arb-token-bridge-ui/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"cheerio": "^1.0.0-rc.12",
"dayjs": "^1.11.8",
"ethers": "^5.6.0",
"exponential-backoff": "^3.1.1",
"graphql": "^16.8.1",
"lodash-es": "^4.17.21",
"next": "^14.2.12",
Expand Down
11 changes: 11 additions & 0 deletions packages/arb-token-bridge-ui/src/util/ExponentialBackoffUtils.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
import { backOff as _backOff, BackoffOptions } from 'exponential-backoff'

const backoffOptions: BackoffOptions = {
startingDelay: 1_000,
timeMultiple: 1.5
}

export function backOff<T>(request: () => Promise<T>): Promise<T> {
return _backOff(request, backoffOptions)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to turn on full jitter?
https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/

i think we do?

}

export function wait(ms: number) {
return new Promise(resolve => setTimeout(resolve, ms))
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export function fetchETHWithdrawalsFromEventLogs({
l2Provider: Provider
}) {
if (typeof receiver === 'undefined') {
return []
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise the type would be never[]

return Promise.resolve([])
}

// funds received by this address
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {
} from './fetchTokenWithdrawalsFromEventLogs'
import { getNonce } from '../AddressUtils'
import { fetchL2Gateways } from '../fetchL2Gateways'
import { wait } from '../ExponentialBackoffUtils'
import { backOff, wait } from '../ExponentialBackoffUtils'

async function getGateways(provider: Provider): Promise<{
standardGateway: string
Expand Down Expand Up @@ -101,7 +101,7 @@ export async function fetchTokenWithdrawalsFromEventLogsSequentially({
}

const gateways = await getGateways(provider)
const senderNonce = await getNonce(sender, { provider })
const senderNonce = await backOff(() => getNonce(sender, { provider }))

// sender queries; only add if nonce > 0
if (senderNonce > 0) {
Expand Down Expand Up @@ -130,7 +130,7 @@ export async function fetchTokenWithdrawalsFromEventLogsSequentially({

const currentPriorityResults = await Promise.all(
currentPriorityQueries.map(query =>
fetchTokenWithdrawalsFromEventLogs(query.params)
backOff(() => fetchTokenWithdrawalsFromEventLogs(query.params))
)
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import { Withdrawal } from '../../hooks/useTransactionHistory'
import { attachTimestampToTokenWithdrawal } from './helpers'
import { WithdrawalInitiated } from '../../hooks/arbTokenBridge.types'
import { fetchTokenWithdrawalsFromEventLogsSequentially } from './fetchTokenWithdrawalsFromEventLogsSequentially'
import { wait } from '../ExponentialBackoffUtils'
import { backOff, wait } from '../ExponentialBackoffUtils'

export type FetchWithdrawalsParams = {
sender?: string
Expand Down Expand Up @@ -84,12 +84,16 @@ export async function fetchWithdrawals({
console.log('Error fetching withdrawals from subgraph', error)
}

const ethWithdrawalsFromEventLogs = await fetchETHWithdrawalsFromEventLogs({
receiver,
fromBlock: toBlock + 1,
toBlock: 'latest',
l2Provider: l2Provider
})
const ethWithdrawalsFromEventLogs = await backOff(() =>
fetchETHWithdrawalsFromEventLogs({
receiver,
// not sure why eslint is treating "toBlock" as "number | undefined" here
// even though typescript recognizes it as "number"
fromBlock: toBlock ?? 0 + 1,
toBlock: 'latest',
l2Provider: l2Provider
})
)

await wait(2_000)

Expand Down
5 changes: 5 additions & 0 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -7545,6 +7545,11 @@ expect@^29.0.0, expect@^29.5.0:
jest-message-util "^29.5.0"
jest-util "^29.5.0"

exponential-backoff@^3.1.1:
version "3.1.1"
resolved "https://registry.yarnpkg.com/exponential-backoff/-/exponential-backoff-3.1.1.tgz#64ac7526fe341ab18a39016cd22c787d01e00bf6"
integrity sha512-dX7e/LHVJ6W3DE1MHWi9S1EYzDESENfLrYohG2G++ovZrYOkm4Knwa0mc1cn84xJOR4KEU0WSchhLbd0UklbHw==

express@^4.17.3:
version "4.21.0"
resolved "https://registry.yarnpkg.com/express/-/express-4.21.0.tgz#d57cb706d49623d4ac27833f1cbc466b668eb915"
Expand Down
Loading