[Idea] Automation: Comment with PR checklist when commits are pushed

[jaclyn-taroni]

Problem

One of the things identified as an issue in #228 is that it can be hard to verify that all conditions in the PR checklist are met when pull requests go through multiple iterations, both as a pull request author and a reviewer. It's possible that a new checklist could help a pull request author ensure that all conditions are met and give the reviewer confidence.

Idea

What if we automate a comment with a blank version of the pull request checklist? The pull request author would then edit the comment to make sure all checklist items were satisfied. The comment would get posted when commits are pushed.

Here's something I found that might be helpful: https://github.com/unsplash/comment-on-pr

If #239 happens, this could get very chatty very fast! Also, if we automate things to the point where everything other than the content changes are pretty well covered, this may be less valuable.

[cansavvy]

I'm a tad worried about this diluting the conversations that happen on the PRs (this is highly dependent on how often people push commits. I have a tendency to push commits a lot, but I can cut back), but I do like the idea, it doesn't sound like its too hard to implement so we could give it a try and abandon it if we find it to be a nuisance.

[jaclyn-taroni]

Yes, agreed, one would have to be cognizant of not pushing after every commit. The idea would be to push when you're ready to review the checklist again with this one.

[jashapiro]

We could make it trigger on review requests?

https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows#pull_request

on:
  pull_request:
    types: [ review_requested ]

[jaclyn-taroni]

We could make it trigger on review requests?

This seems like exactly what we want, yes.

[cansavvy]

Is this something we still would like to try out? Or do we want to see how things are with the more recent automations first?

[jaclyn-taroni]

My inclination is that the automated steps and the checklist are not mutually exclusive, and therefore this could still be useful in theory. But I agree with waiting to see how things are going in practice!