Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Fix unintentional ignoring of console errors #19053

Merged
merged 1 commit into from
Oct 18, 2023

Conversation

jelly
Copy link
Member

@jelly jelly commented Jul 3, 2023

Commit ccab549 broke the checking of browser errors, as with an
unset $TEST_ALLOW_JOURNAL_MESSAGES (which is the case in our CI) it
was adding the empty string as allowed pattern, which always matches.

Ignore the React errors that happen with our current code, and add some
explanations. We have so many of them that we can't fix all of them in
one go, but we have to put a stop to introducing new ones. Some of them
are test specific.


This PR originally had a lot of extra commits with aria updates. I split them out into PR #19496, but it's likely that they won't land, and we should ignore the aria warnings for the time being.

@jelly jelly added the no-test For doc/workflow changes, or experiments which don't need a full CI run, label Jul 3, 2023
@jelly jelly changed the title ntest: don't ignore unknown console.error's test: don't ignore unknown console.error's Jul 3, 2023
@jelly
Copy link
Member Author

jelly commented Jul 3, 2023

error: Tracer failed: "{"problem":"access-denied","exit_status":null,"exit_signal":null,"message":"Not permitted to perform this action."}", data: "undefined"

This needs to be allowed.

@jelly jelly force-pushed the gate-on-pf-warnings branch 4 times, most recently from 220dce5 to a8c5bdf Compare July 4, 2023 14:11
@jelly
Copy link
Member Author

jelly commented Jul 4, 2023

We are left with:

  • TestSuperuser.testMultipleBridgeConfig error: FormSelect requires either an id or aria-label to be specified
  • TestApps.testWithUrlRoot
error: 

# Result testWithUrlRoot (__main__.TestApps.testWithUrlRoot) failed
  • TestStorageLuks.testLuks/testNoFs error: Text input: Text input requires either an id or aria-label to be specified

@jelly jelly force-pushed the gate-on-pf-warnings branch from 49c3c5f to 109c0fc Compare July 5, 2023 10:37
@jelly
Copy link
Member Author

jelly commented Jul 5, 2023

> error: Text input: Text input requires either an id or aria-label to be specified
> warning: transport closed: disconnected
Traceback (most recent call last):
  File "/work/bots/make-checkout-workdir/test/common/testlib.py", line 1598, in tearDown
    self.check_browser_errors()
  File "/work/bots/make-checkout-workdir/test/common/testlib.py", line 1868, in check_browser_errors
    raise Error(UNEXPECTED_MESSAGE + "browser errors:\n" + log)
testlib.Error: FAIL: Test completed, but found unexpected browser errors:
error: Text input: Text input requires either an id or aria-label to be specified

# Result testCreate (__main__.TestTimers.testCreate) failed
# testBasic (__main__.TestApps.testBasic)
Failed to kill unit packagekit.service: Unit packagekit.service not loaded.
Failed to reset failed state of unit packagekit.service: Unit packagekit.service not loaded.
rm: cannot remove '/etc/systemd/system/dnf-automatic*': No such file or directory

DevTools listening on ws://127.0.0.1:9487/devtools/browser/45c78bab-0571-4b02-aadc-fe8dd6e9d2cc
[0705/114929.437608:WARNING:sandbox_linux.cc(393)] InitializeSandbox() called with multiple threads in process gpu-process.
warning: source_date_epoch_from_changelog set but %changelog is missing
warning: source_date_epoch_from_changelog set but %changelog is missing
> log: PackageKit went away from D-Bus
> log: PackageKit went away from D-Bus
> warning: PackageKit went away during transaction /9_ceabddcd : {"command":"close","channel":"1:5!5"}
> error: 
> warning: transport closed: disconnected
Traceback (most recent call last):
  File "/work/bots/make-checkout-workdir/test/common/testlib.py", line 1598, in tearDown
    self.check_browser_errors()
  File "/work/bots/make-checkout-workdir/test/common/testlib.py", line 1868, in check_browser_errors
    raise Error(UNEXPECTED_MESSAGE + "browser errors:\n" + log)
testlib.Error: FAIL: Test completed, but found unexpected browser errors:
error: 

@jelly jelly mentioned this pull request Jul 5, 2023
@martinpitt martinpitt added blocked Don't land until something else happens first (see task list) needs-rebase labels Oct 17, 2023
@martinpitt
Copy link
Member

I locally rebased this PR. Many of the commits are already in main, and I sent PR #19496 for some more. After that, let's rebase this and review what's left. @jelly says Garrett has advised against adding unnecessary aria-labels.

@jelly
Copy link
Member Author

jelly commented Oct 17, 2023

I'd suggest we let this run with just fedora-38/devel after a rebase and see what the damage is

@martinpitt martinpitt removed blocked Don't land until something else happens first (see task list) needs-rebase labels Oct 17, 2023
@martinpitt
Copy link
Member

I rebased this and cleaned this up. See description for the aria stuff, let's handle that separately. Doing an initial run, and then I'll just add the aria bits to allowed messages. Then we have a nice laundry list of cleanups, and avoid introducing new ones.

@jelly
Copy link
Member Author

jelly commented Oct 17, 2023

I rebased this and cleaned this up. See description for the aria stuff, let's handle that separately. Doing an initial run, and then I'll just add the aria bits to allowed messages. Then we have a nice laundry list of cleanups, and avoid introducing new ones.

Ack, I agree with leaving the aria stuff for a later cleanup.

@martinpitt
Copy link
Member

The first run is quite a mouthful. I went through with a coarse-grained grep | sort and ignore them wholesale. I suggest to land this ASAP, and then we can parallelize the actual fixes.

@martinpitt
Copy link
Member

I also changed the approach for fixing the "" handling. Your original patch did not work very well, and it also turned "search" into "match" semantics.

@martinpitt martinpitt changed the title test: don't ignore unknown console.error's test: Fix unintentional ignoring of console errors Oct 17, 2023
@martinpitt martinpitt removed the no-test For doc/workflow changes, or experiments which don't need a full CI run, label Oct 17, 2023
@martinpitt martinpitt force-pushed the gate-on-pf-warnings branch 2 times, most recently from debecdb to dedc93e Compare October 17, 2023 15:50
@martinpitt martinpitt marked this pull request as ready for review October 17, 2023 17:36
martinpitt
martinpitt previously approved these changes Oct 17, 2023
Copy link
Member

@martinpitt martinpitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is sort of weird.. Mostly my changes now, but @jelly 's PR. I formally approve, but @jelly please re-review and land if you are happy.

Copy link
Member Author

@jelly jelly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Commit ccab549 broke the checking of browser errors, as with an
unset `$TEST_ALLOW_JOURNAL_MESSAGES` (which is the case in our CI) it
was adding the empty string as allowed pattern, which always matches.

Ignore the React errors that happen with our current code,  and add some
explanations. We have so many of them that we can't fix all of them in
one go, but we have to put a stop to introducing new ones. Some of them
are test specific.

Co-Authored-By: Martin Pitt <[email protected]>
@martinpitt
Copy link
Member

Argh, one more, due to a stupid typo. Fixed. It's blindingly obvious, so re-approving.

@martinpitt martinpitt merged commit 5ac51ec into cockpit-project:main Oct 18, 2023
35 checks passed
@martinpitt martinpitt mentioned this pull request Oct 18, 2023
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants