Support for extra pre-merge checks

[RalfJung]

One key feature of bors/homu for GitHub is that one can set up CI such that the pre-merge (post-approval) checks performed by the bot are more extensive than the regular PR checks. This is useful in particular when there is not enough CI capacity to run the entire test suite for each push to an MR. bors/homu achieve that by using a staging branch instead of relying on PR CI to determine if a PR is good; CI config can then be set up to perform extra checks on pushes to the staging branch. (The concept of a staging branch also nicely solves #111.)

It looks like marge-bot has no way to achieve the same, i.e., if I want to have extensive and expensive pre-landing checks, I have to set up the same checks to be run for each push to each MR. Is that right? In our case that would go beyond our available CI capacity, sadly.

[Jellby]

You could configure the expensive tests to be run only if some CI variable is specified, or only if the pipeline source is api or web...

You still need some logic from marge-bot, to identify when a pipeline is not "good enough" and trigger good pipeline, and this is done in #114

[RalfJung]

You could configure the expensive tests to be run only if some CI variable is specified, or only if the pipeline source is api or web...

I looked through the marge-bot options and found no way to set a CI variable for marge-bot-triggred jobs; does such an option exist?

Pipeline source could work as a distinguisher though; this would mean marge would have to always trigger a new pipeline via the API before merging an MR to ensure that the extra jobs get run.

Somehow this still seems much more complicated than homu/bors with their staging branch.^^

[Jellby]

I looked through the marge-bot options and found no way to set a CI variable for marge-bot-triggred jobs; does such an option exist?

I don't think it does. I meant that it should be possible to achieve the goal without a staging branch, but marge-bot still needs to be updated.

[snim2]

Maybe I have misunderstood, but I think you could just do this with GitLab pipeline rules and the predefined variables and ignore Marge altogether.

I think you want something like this:

1. Has the pipeline been triggered by the Marge bot (needs GITLAB_USER_LOGIN)?
2. If "no" to 1. assume the pipeline is for a feature branch...
3. If "yes" to 1., is it running on the default branch (or the target branch of an MR) (needs CI_COMMIT_BRANCH and maybe some of the MR variales depending on your CI config)?
4. If "yes" to 2. run the post-merge jobs
5. If "yes" to 1. and "no" to 2, run the pre-merge jobs

[RalfJung]

Has the pipeline been triggered by the Marge bot (needs GITLAB_USER_LOGIN)?

So marge will always trigger a new pipeline, and not rely on the result of the previous MR/branch pipeline run if that one is green and up-to-date?

[snim2]

My understanding is that Marge will always trigger a new pipeline on the target branch after the merge, but usually Marge rebases and force-pushes to the source branch before the merge, and so this would normally trigger a pipeline.

[RalfJung]

Well, "after the merge" is too late, the point is to make master always green after all. ;)

If the pre-merge pipeline is only "usually" created it's not really something we cold rely on.

[Jellby]

At least with my setup, Marge doesn't trigger a pipeline unless she's asked to. In normal circumstances, she just waits for the existing pipeline (triggered when the developer created the MR or pushed to the source branch) and accepts the MR (which then updates the target branch and triggers another pipeline).

... and if the MR is outdated, she pushes an update to the source branch, which also triggers a MR pipeline.

[RalfJung]

In normal circumstances, she just waits for the existing pipeline (triggered when the developer created the MR or pushed to the source branch) and accepts the MR (which then updates the target branch and triggers another pipeline).

Yeah, that doesn't really work for my use-case. The target branch pipeline is not relevant here as the entire point is to make sure it doesn't fail to begin with, so the MR/feature-branch pipeline has to do all the testing -- but only after we approved the MR.

[snim2]

I'm not sure whether it's just me, but I'm a bit confused here.

If you set Marge to add a trailer on the MR commit messages, then the bot will always force-push to the source branch. That way you know which pipeline is run just before the merge, because it's the one started by the bot.

[RalfJung]

If you set Marge to add a trailer on the MR commit messages, then the bot will always force-push to the source branch. That way you know which pipeline is run just before the merge, because it's the one started by the bot.

Well if we can rely on marge to trigger a pipeline then we can use that pipeline for our extra checks -- that's what I wanted to figure out. :)

This is much harder to wrap my head around than a staging branch, but it sounds like it could work.

[snim2]

Personally, I'd be inclined to set up a spike repo with the same .gitlab-ci.yml but replace all the scripts with no-ops, and see how you go. That way you can try out different marge-bot configuration options without fear of breaking anything that matters. (But then I'm always very conservative about these infrastructure changes!)

[RalfJung]

Certainly we'd do a testrun first, yeah.

But it'll be a while until we get there, we'll first need to find a way such that #111 does not affect us any more. Until earlier this week I assumed there'd be a staging branch, so I need to look into getting CI to work in forks. ;)

[nithyashree675]

#340 could help ensure your pipeline is run (atleast once) only when assigned to marge-bot, something like

if (env.gitlabMergedByUser == "marge-bot") {
                functions.approversCheck()
                functions.deployToStaging()
} 

[morguldir]

@nithyashree675 it's nice to see some activity, that looks like a usable solution. Does this activity mean that someone will be looking at other merge requests? Otherwise it might be time to pass the torch on to another organization as mentioned in #295.