Stdout hijacking in api.py is not thread safe

[randomstuff]

See tomv564/pyls-mypy#19, AFAIU mypy api.py relying on sys.stdout overriding is not thread-safe and causes issues when sys.stout and sys.stderr are used concurrently.

[gvanrossum]

This is a good catch. It may be a bit tricky to fix, but we’re open to PRs. There probably aren’t too many places where stdout or street are used, but passing something in may require a bunch of messy signature changes.

[JukkaL]

It would be better to explicitly pass the stdout and stderr objects instead of modifying them in sys. Would you like to contribute a fix? This should be a pretty easy one.

Here's a summary of what's involved for anybody interested in helping with this:

• Don't modify sys.stdout and sys.stderr in mypy.api.
• Instead, pass relevant StringIO objects to mypy.main.main as arguments. These can default to sys.stdin and sys.stderr in main.
• Use these arguments when we generate error messages.
  • No need to update dmypy, stubgen and logging to use these, since they aren't relevant when using api.py.

[elkhadiy]

Erm... Well it seems Github happily shows any reference of an issue from any fork whatsoever... that's good to know.

I wanted to wait for tomorrow so my colleague could test it on his setup since he was more "lucky" than me in getting the race conditions to "work" on his machine(tm). I guess nothing's wrong in putting the PR up for review now.

[emmatyping]

@elkhadiy, take the time you need to make a good change set. Don't worry we won't jump on you :)

[AnOctopus]

What remains to be done about this issue? It causes a lot of problems on my setup (using pyls-mypy and spacemacs) and I'd be willing to do some work to fix it but the changes @elkhadiy made on their fork seem to already cover the needed changes.

[emmatyping]

@AnOctopus I believe the fork was committed to but a PR was never opened. If you would like to fix this, please feel free to open a PR! (Or perhaps @elkhadiy is still interested?)

[AnOctopus]

PR #6750 opened.