- Thursday, November 9 2023
Haskell reliability was the theme of the call.
Stability of recent GHC major versions, in particular.
I have noticed a disconnect in the general vibe felt among GHC users compared to the concrete issues reported upstream. Some people report entire major versions as unusable, and yet as I watch GHC HQ triage issues, I see them promptly solve nearly every problem that is brought to their attention. Something seems to be missing. Can we bridge the information gap somehow? Is it just a faulty perception, or are important issues not getting raised upstream?
Jack suggested it would be nice if there was a way to encourage users to try things out and pass reports back upstream. Ironically, the very next day, Cabal maintainers decided to do a very similar thing, which culminated in their [call for Windows testers][cabal-win]. But for Cabal, the request is clear: run these tests against that Cabal version. GHC doesn't have the same infrastructure yet.
That is one argument for ensuring bindist tests are reliable. But that requires making tests reliable, full stop. And there's only so much GHC HQ can do if they don't have access to whatever exotic architecture on which a problem arises.
Another option is to turn https://gitlab.haskell.org/ghc/head.hackage into a user-facing utility: Download this tarball and run the tests and report the results. I personally can imagine having a "GHC@Home" program that packages all of this up real pretty. It would be great to get a really wide set of data on running GHC on real environments in all sorts of places. It wouldn't replace the testsuite, but would augment it. The testsuite could be 100% rock-solid and it would still benefit from real-world reports.
Another project to put in my "maybe" list...
The final option we discussed was to make GHC Nightlies reliable, and then give users a clear set of steps to try building their packages with it and reporting results. This would work especially well for smaller packages with a smallish set of dependencies. But don't worry, making GHC Nightlies reliable was already on my "maybe" list.
The mic drop of the year award goes to... was it Jack or Andrea? I already forgot! On the subject of GHC triage, they said it would be nice to see a summary of the weekly triage meetings. That was such a good idea that I immediately resolved to start doing so. The first one will be published soon.
[cabal-win]: https://discourse.haskell.org/t/cabal-is-looking-for-qa-testers-on-the-windows-platform/8103)