Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

manifest: move pool repo to lockfile-repos #355

Merged
merged 1 commit into from
Apr 29, 2020

Conversation

jlebon
Copy link
Member

@jlebon jlebon commented Apr 17, 2020

The pool repo isn't semantically an ordinary like the others. It
contains past and present locked packages used in any Fedora CoreOS
stream. Mark it as a lockfile repo instead to tell rpm-ostree to only
use this repo for fetching locked packages and never to try to freely
depsolve with it on.

@jlebon
Copy link
Member Author

jlebon commented Apr 17, 2020

Requires: coreos/rpm-ostree#2058

Copy link
Member

@dustymabe dustymabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jlebon jlebon force-pushed the pr/lockfile-repo-pool branch from 17e00eb to d7178db Compare April 22, 2020 19:00
@jlebon jlebon marked this pull request as ready for review April 22, 2020 19:00
@jlebon
Copy link
Member Author

jlebon commented Apr 22, 2020

OK, this is ready now! Though it requires coreos/coreos-assembler#1387 first. (Well, technically it doesn't require it, but they'll work together in the context of the lockfile updater.)

The pool repo isn't semantically an ordinary like the others. It
contains past and present locked packages used in any Fedora CoreOS
stream. Mark it as a lockfile repo instead to tell rpm-ostree to only
use this repo for fetching locked packages and never to try to freely
depsolve with it on.
@jlebon jlebon force-pushed the pr/lockfile-repo-pool branch from d7178db to 9824856 Compare April 28, 2020 21:13
@jlebon
Copy link
Member Author

jlebon commented Apr 28, 2020

OK, I kept this localized to manifest.yaml for now. We'll move it to fedora-coreos.yaml once we have coreos/rpm-ostree#2071 so we can drop repos entirely from the production branches' manifest.yaml files.

Once this merges, I'm going to do the same thing to next-devel and introduce back lockfiles there.

Copy link
Member

@dustymabe dustymabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jlebon jlebon merged commit 285fb34 into coreos:testing-devel Apr 29, 2020
@jlebon jlebon deleted the pr/lockfile-repo-pool branch April 29, 2020 13:44
jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request Apr 29, 2020
The pool repo isn't semantically an ordinary like the others. It
contains past and present locked packages used in any Fedora CoreOS
stream. Mark it as a lockfile repo instead to tell rpm-ostree to only
use this repo for fetching locked packages and never to try to freely
depsolve with it on.

(Cherry-picked from coreos#355.)
jlebon added a commit that referenced this pull request Apr 29, 2020
The pool repo isn't semantically an ordinary like the others. It
contains past and present locked packages used in any Fedora CoreOS
stream. Mark it as a lockfile repo instead to tell rpm-ostree to only
use this repo for fetching locked packages and never to try to freely
depsolve with it on.

(Cherry-picked from #355.)
@jcajka
Copy link
Contributor

jcajka commented Apr 30, 2020

This breaks non-intel as there are no lock files there yet. This effectively disabled the fedora-coreos-pool repo there. @jlebon do you have any advice how to proceed?

@jlebon
Copy link
Member Author

jlebon commented Apr 30, 2020

This breaks non-intel as there are no lock files there yet.

Ahh right, yeah. All builds are unlocked there, but I'm guessing you're not getting a depsolve without the pool because of #335 (comment), right?

Are you getting a successful compose with:

diff --git a/manifest-lock.overrides.s390x.yaml b/manifest-lock.overrides.s390x.yaml
new file mode 100644
index 0000000..637eb4e
--- /dev/null
+++ b/manifest-lock.overrides.s390x.yaml
@@ -0,0 +1,3 @@
+packages:
+  crypto-policies:
+    evra: 20191128-5.gitcd267a5.fc32.noarch

?

This effectively disabled the fedora-coreos-pool repo there. @jlebon do you have any advice how to proceed?

The right fix is adding lockfiles for all the arches :). But yeah, we're not there yet.

For now though, I think the right fix is simply maintaining overrides for all the arches, not just x86_64. Otherwise, even if you did get a depsolve without the pool, you'd still be missing out on all the other packages we fast-track.

Will open a PR!

jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request Apr 30, 2020
With coreos#355, the pool
is now used for fetching lockfiles only. This broke multi-arch builds
though because although there are no lockfiles there and it's currently
not possible to get a depsolve without at least locking crypto-policies:

coreos#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
multi-arch builds.
jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request Apr 30, 2020
With coreos#355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles there and it's currently
not possible to get a depsolve without at least locking crypto-policies:

coreos#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
multi-arch builds.
jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request Apr 30, 2020
With coreos#355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles, it's currently not
possible to get a depsolve without at least locking crypto-policies:

coreos#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
multi-arch builds.
jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request Apr 30, 2020
With coreos#355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles, it's currently not
possible to get a depsolve without at least locking crypto-policies:

coreos#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
builds there.
@jlebon
Copy link
Member Author

jlebon commented Apr 30, 2020

#374

@jcajka
Copy link
Contributor

jcajka commented Apr 30, 2020

That should be it. I have not tried that on the "lock" level yet, just the repo level. I have proposed in the past to have not arch-full lock as if I correctly understand that it is really more of a "override" list and currently there is no arch dependent override(correct me if I'm wrong). It seems to me as rather tedious and unnecessary job to maintain locks in current format. IMO it seems that this should be done on the "override" repo side, only provide packages that are needed there.

@jlebon
Copy link
Member Author

jlebon commented Apr 30, 2020

I have proposed in the past to have not arch-full lock as if I correctly understand that it is really more of a "override" list and currently there is no arch dependent override(correct me if I'm wrong).

I don't think it's come up yet, but we want to retain that flexibility. (This is similar to how e.g. the stream metadata can in theory support arch-dependent overrides).

It seems to me as rather tedious and unnecessary job to maintain locks in current format.

Well, there are two bits; the base lockfiles and the overrides. The former is automated (well, not right now because of some issues, but it normally is), while the latter is manual, by definition.

IMO it seems that this should be done on the "override" repo side, only provide packages that are needed there.

I'm not sure if I fully understand what you mean here. Are you suggesting leaving the base unlocked and only maintain an overrides repo with e.g. priority=1 or something?

(Though in general, yes it's possible to have "locked package sets" on the yum repo side and that would solve issues and create others. We went with what best matches the current state of Fedora today. See somewhat related discussions in coreos/rpm-ostree#415.)

jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request May 1, 2020
With coreos#355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles, it's currently not
possible to get a depsolve without at least locking crypto-policies:

coreos#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
builds there.
jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request May 1, 2020
With coreos#355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles, it's currently not
possible to get a depsolve without at least locking crypto-policies:

coreos#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
builds there.
jlebon added a commit that referenced this pull request May 1, 2020
With #355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles, it's currently not
possible to get a depsolve without at least locking crypto-policies:

#335 (comment)

While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
builds there.
jlebon added a commit to jlebon/fedora-coreos-config that referenced this pull request Jul 14, 2021
This is common to all streams, so we can move it here. I had initially
kept it in `manifest.yaml` because of:

coreos#355 (comment)
jlebon added a commit that referenced this pull request Jul 14, 2021
This is common to all streams, so we can move it here. I had initially
kept it in `manifest.yaml` because of:

#355 (comment)
HuijingHei pushed a commit to HuijingHei/fedora-coreos-config that referenced this pull request Oct 10, 2023
This is common to all streams, so we can move it here. I had initially
kept it in `manifest.yaml` because of:

coreos#355 (comment)
HuijingHei pushed a commit to HuijingHei/fedora-coreos-config that referenced this pull request Oct 10, 2023
This is common to all streams, so we can move it here. I had initially
kept it in `manifest.yaml` because of:

coreos#355 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants