test: fix flaky e2e #2158
test: fix flaky e2e #2158
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
| @@ -50,7 +50,7 @@ describe('Approve token for deposit', () => { | |||
| * If confirm spending fails, test is still considered to be passing by Cypress | |||
| * We add another check to make sure the test fails if needed | |||
| */ | |||
| cy.wait(10_000) | |||
| cy.wait(25_000) | |||
There was a problem hiding this comment.
Orbit chains may take longer, sometimes the tests fail because they don't have enough time to run
There was a problem hiding this comment.
can we only make it 25_000 when it's running for an Orbit chain?
There was a problem hiding this comment.
I think it's good to have it for both, I'm only guessing it's for Orbit only as I've observed in the videos. But to be sure I'd leave it for both.
There was a problem hiding this comment.
Anyway, 15 extra seconds doesn't matter because we always wait for withdrawal e2e to finish at the end, they take a bit longer
| timeout: 60_000, | ||
| interval: 500 | ||
| }) | ||
| cy.findTransactionInTransactionHistory(txData) |
There was a problem hiding this comment.
waitUntil doesn't work properly here because it needs to return a result. I've moved timeout to the method itself so we always try for 120_000
| .then(parseFloat) | ||
| .should( | ||
| 'be.gt', | ||
| cy.findByLabelText( |
There was a problem hiding this comment.
invoke doesn't follow the retry mechanism, so the default timeout is not respected. With this change we apply the default timeout and fail tests less often.
There was a problem hiding this comment.
you can also add a { timeout: 10_000 } as the second arg of findByLabelText
There was a problem hiding this comment.
We have a default timeout of 20_000 for all commands now. Currently we used synpress's default timeout of 4_000 for all commands. This is not enough and caused some flakiness.
With 20_000 we don't really make the tests longer, they will only fail longer if they do.
| @@ -103,7 +103,7 @@ describe('Withdraw ERC20 Token', () => { | |||
| }) | |||
|
|
|||
| context('should show clickable withdraw button', () => { | |||
| cy.findMoveFundsButton().click() | |||
| cy.startTransfer({ shouldConfirmInMetamask: false }) | |||
There was a problem hiding this comment.
startTransfer has wait methods to ensure there are no race conditions. For withdrawals we don't want to confirm in metamask popup because we have our dialogs that pop up before that.
| @@ -226,7 +226,8 @@ export default defineConfig({ | |||
| }, | |||
| baseUrl: 'http://localhost:3000', | |||
| specPattern: tests, | |||
| supportFile: 'tests/support/index.ts' | |||
| supportFile: 'tests/support/index.ts', | |||
| defaultCommandTimeout: 20_000 | |||
There was a problem hiding this comment.
By default it's set to 4_000. It's not enough given a lot of things are awaited for (e.g. balance changes after transfer etc), so we bump it by a significant amount.
This won't make checks any longer than they are now, only if it fails it will take a bit longer to fail. But it's a good tradeoff to eliminate flakiness.
packages/arb-token-bridge-ui/tests/e2e/specs/withdrawNativeToken.cy.ts
Outdated
Show resolved
Hide resolved
No description provided.