Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Fix pytest race in test_spawn_broken_pipe() #19470

Merged
merged 1 commit into from
Oct 13, 2023

Conversation

martinpitt
Copy link
Member

@martinpitt martinpitt commented Oct 12, 2023

The static time.sleep(0.1) was racy: On slow architectures like Debian mips [1] it repeatedly fails, and it even sometimes fails on GitHub actions.

Keep writing until the transport closes, so that we are guaranteed to eventually run into a BrokenPipeError.

Fixes #19469

[1] https://buildd.debian.org/status/logs.php?pkg=cockpit&ver=302-1&arch=mips64el


Failures on GitHub: example 1, example 2

@martinpitt martinpitt added the no-test For doc/workflow changes, or experiments which don't need a full CI run, label Oct 12, 2023
@martinpitt martinpitt removed the no-test For doc/workflow changes, or experiments which don't need a full CI run, label Oct 12, 2023
@martinpitt
Copy link
Member Author

(I realized I don't actually want no-test, as this test runs during rpm/deb package builds)

Copy link
Member

@allisonkarlitskaya allisonkarlitskaya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to merge this if the other approach doesn't work.

with open(f'/proc/{pid}/stat') as f:
if f.read().split()[2] == 'Z':
break
time.sleep(0.1)
Copy link
Member

@allisonkarlitskaya allisonkarlitskaya Oct 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm generally okay with this approach, but you could also do something like:

# The process will exit soon — try writing to it until a write fails.
# Note: it's important that we observe .write() failing, so we do sync
# sleep to avoid being notified about the process exiting.
while not transport.is_closing():
    transport.write(b'x')
    time.sleep(0.1)

no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nice idea -- catch it from the write side instead of the process exit side. Seems to work fine!

@allisonkarlitskaya
Copy link
Member

Interesting: we thought long and hard about the race failing when the process exited too quickly (before the registration of the exit handler) but never considered the other direction :)

The static `time.sleep(0.1)` was racy: On slow architectures like Debian
mips [1] it repeatedly fails, and it even sometimes fails on GitHub
actions.

Keep writing until the transport closes, so that we are guaranteed to
eventually run into a BrokenPipeError.

Fixes cockpit-project#19469

[1] https://buildd.debian.org/status/logs.php?pkg=cockpit&ver=302-1&arch=mips64el
@allisonkarlitskaya allisonkarlitskaya merged commit 6f37c73 into cockpit-project:main Oct 13, 2023
37 checks passed
@martinpitt martinpitt deleted the spawn-test branch October 13, 2023 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-blocker Targetted for next release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test/pytest/test_peer.py::test_spawn_broken_pipe is flaky
2 participants