Add a lock feature

[essen]

Adding a lock.mk file with generated repo URL/version should be a good solution to automate this.

[essen]

The idea is that you don't need to do anything to have reproduceable builds. Erlang.mk would create a lock file on compile (only in the top-level erlang.mk) or use it if one already exists. Distclean would not delete it. We could have a "make unlock" command to delete it, perhaps, though rm lock.mk works just as well.

[essen]

The name locks.mk is a little weird. How about deps.mk?

[essen]

Wouldn't it be cool if it was updated every time the dep changed? For example cowboy i develop with symlinks to my local cowlib and ranch repo. If i update something there i usually want to keep track and have to update manually in cowboy. If it's updated at compile time then i can just commit everything in one go. So updating deps would be about modifying the dep commit and then calling make in the top level repo.

This could be either automated or through a separate target. Not sure which is best, have to experiment.

[jpuigm]

What's the status of this? Is there any current development?

I feel like the only way to work around this today is still to manually keep an alternative .packages file with specific commits to lock all dependencies down in order to achieve reproducible builds. However, the option to keep a local .packages file has been removed a while ago, which is stopping me from upgrading to a more recent version of erlang.mk.

[essen]

You can just use "dep_cowboy_commit = 1.0.1" type of lines for now if you want.

This has made little progress in parts because I'm not exactly sure how this should be implemented best. The comments above are just hints. I need feedback as to what people currently do and expect. The other reason is that we need everything else documented/tested before adding too many features (though for this particular ticket, this isn't blocking as the relevant parts are all documented/tested already, it's just priorities).

[essen]

Based on my experience with rebar3 in a customer project, I want this to not be automatic, but rather the file be created when running make lock, and unlocked when running make unlock or when a Makefile has been modified (so that a dep change can be considered); automatic unlock would print a message about it of course.

[essen]

Automatic unlock should not be 100% automatic, I'm sure we can detect when dependencies actually changed, rather than it being just an unrelated change.

[essen]

@dumbbell thoughts

[dumbbell]

So your idea so far is mostly to have this locking be a simple "snapshot" of the dependencies' details at the time of the make lock command. If a dependency is changed in some way, Erlang.mk won't enforce a switch back to the commit/version referenced in lock file. Is that correct?

[essen]

I think if the user went into deps/cowboy and updated manually, the lock file shouldn't be updated unless the user runs make lock again. It's a special case, I don't think it's worth doing automatically.

[dumbbell]

I was thinking of the opposite situation: if the user goes intodeps/cowboy to switch to another commit, would Erlang.mk (in the top-level project) switch Cowboy back to the commit saved in the lock file during the next run of make? I suppose no.

I think Rebar3 does this kind of enforcement and I didn't like it.

[essen]

No Erlang.mk shouldn't go against the will of the user.

[dumbbell]

Ok, I agree.

So the expected use case is around release builds. During development, you don't have any lock file (or if you have one, it doesn't interfere with you messing with dependencies). But before doing a release, you run make lock and commit/tag the result: people getting your project will thus get the same dependencies as you.

[essen]

Yes.

I think it may be useful to have a mechanism so that when you update the Makefile deps, for example adding a dep or changing a dep version, the lock file is updated and the dependency updated. But this can be thought of later.

[essen]

Related: #309