Moving Packages

[Trenly]

Looking at all the manifests, there is an established pattern of Publisher.Package[.SubPackage][.Distro] to ensure that packages which are substantially similar are placed under the same folder in the manifest structure. As I was looking, I found that there are several packages which could be moved to new identifiers to align with the pattern better. One example of this is BraveBrowser -

PackageIdentifier -> Suggested Identifier
BraveSoftware.BraveBrowser
BraveSoftware.BraveBrowser-Beta -> BraveSoftware.BraveBrowser.Beta
BraveSoftware.BraveBrowser-Dev -> BraveSoftware.BraveBrowser.Dev
BraveSoftware.BraveBrowser-Nightly -> BraveSoftware.BraveBrowser.Nightly

Since the pipelines don't yet support multiple changes in a PR, I have a script that can generate the branches required to move each version individually. However, looking at the number of packages that I am proposing updates to, this would result in ~2,300 PR's.

I do not want to flood the repository with PR's, since this would likely overwhelm the moderators and the pipelines even if done in small chunks. Additionally, there is likely to be domain validation issues with some of these packages (See #30688 from when I tested my script).

This leaves me with two main questions -

1. Is it okay / feasible to move these packages? I do know that package moves are intended to be difficult since PackageIdentifiers are intended to be a static field, but I think that in some cases it is warranted. At the very least a process for moving packages should be identified since it is sometimes necessary.
2. If it is okay to move these packages, is there a way to get MSFT's support to approve these changes/prs knowing that these are existing, valid packages and the only information which is changing is metadata related

Thanks to @jedieaston for encouraging me to put my thoughts into an issue

proposed_updates.csv

[vedantmgoyal9]

If it is okay to move these packages, is there a way to get MSFT's support to approve these changes/prs knowing that these are existing, valid packages and the only information which is changing is metadata related

I think repository maintainers can approve but @denelon broke the Publish Pipeline (which is different from Validation Pipeline) once while approving multiple changes in one PR. Reference: #27411 (comment)

[Trenly]

If it is okay to move these packages, is there a way to get MSFT's support to approve these changes/prs knowing that these are existing, valid packages and the only information which is changing is metadata related

I think repository maintainers can approve but @denelon broke the Publish Pipeline (which is different from Validation Pipeline) once while approving multiple changes in one PR. Reference: #27411 (comment)

Yes, I'm aware of that, hence why they would all need to be separate PR's. Because it would be all separate PR's, there would be rate limits when trying to create and approve them all. The creation rate limit isn't an issue, but if there are issues with approving them or with the pipelines, it could easily fill the repository with a lot of PR's and I don't want to cause any confusion, backups, or breakage. I don't know how MSFT has the pipelines set up either, or how they charge internally, but spinning up several hundred runs of a pipeline at the same time could cause it to scale out, and I also don't want to force any additional cost on MSFT if it can be avoided. I'm just trying to be considerate and do it in a way that is as agreeable as possible

[vedantmgoyal9]

It may not be a good suggestion but we can do one (or more) packages per day depending upon how many versions it has inside it.

Also, the v1.1 schema will be coming soon and it will have support for release channels so some packages like Brave Browser (the example you mentioned), and other packages can be avoided for now as we will move them afterwards.

[jedieaston]

This will become more of an issue when 1.1 comes out. There are tons of packages where we changed the version numbers to match the ARP table when they aren't the "real" version numbers. Those will have to be rolled back (with ARP data added to the new keys).

JetBrains, Python, Jackett, AWS CLI, several of the KDE packages, .NET Core (all of them), and probably a bunch of other ones. We should probably start coming up with a strategy to figure that out, I've just been putting it off since there's no roadmap on microsoft/winget-cli#1073 being implemented for list/upgrade.

[Trenly]

This will become more of an issue when 1.1 comes out. There are tons of packages where we changed the version numbers to match the ARP table when they aren't the "real" version numbers. Those will have to be rolled back (with ARP data added to the new keys).

JetBrains, Python, Jackett, AWS CLI, several of the KDE packages, .NET Core (all of them), and probably a bunch of other ones. We should probably start coming up with a strategy to figure that out, I've just been putting it off since there's no roadmap on microsoft/winget-cli#1073 being implemented for list/upgrade.

I can’t agree more. It seems like there will be a large number of versions that need to be moved / updated / changed once the 1.1 schema is released.

I think this truly highlights the need for #125 to become a priority not only for the validation pipeline but also the publish pipeline.

---

Regardless, I think moving some of these packages will help identify packages that 1.1.0 schema changes will need to be made to, since it will be easier to fetch all the folders that contain sub-package or distro folders and then that list can be used to update the packages once 1.1.0 is released

[vedantmgoyal9]

We should start working on #125 by first allowing multiple metadata-only changes in a single PR.

[Trenly]

@denelon - Not sure if this made it to your inbox without the Needs-Triage label; Any inputs you have here would be appreciated

[denelon]

We do have some concerns for users who have built "export" files related to moving packages. I'll bring this Issue up with the team to start looking at as we are closer to opening up support for the 1.1 schema.

[vedantmgoyal9]

The creation rate limit isn't an issue

@Trenly It is an issue. You cannot create more than 200 PRs within an hour. You can see the error I got after opening 200 PRs in one go.

[Trenly]

The creation rate limit isn't an issue

@Trenly It is an issue. You cannot create more than 200 PRs within an hour. You can see the error I got after opening 200 PRs in one go.

Thats IF you’re automating it. I’m not planning on automating, so the 200 limit isn’t an issue

[vedantmgoyal9]

@Trenly I didn't do Discord because Canary and Development are channels of the product Discord.

[Trenly]

@Trenly I didn't do Discord because Canary and Development are channels of the product Discord.

#30732 (comment)

it will be easier to fetch all the folders that contain sub-package or distro folders and then that list can be used to update the packages once 1.1.0 is released

[vedantmgoyal9]

Ok, so I will be doing all of them, thanks for the reference. Actually, I was keeping some packages aside for the v1.1 schema.

[jedieaston]

Can we put a halt to this for a little while? We still haven't got any good ideas on how to make this more efficient, and flooding the queue with rename PRs is blocking us from seeing ones that people actually need approved.

[vedantmgoyal9]

I am not doing any other packages now. There were few versions of OneDrive and Teams left inside the old package identifier because at that time, the limit of creation of PRs was exhausted. Once OneDrive ones would get merged, I would be creating PRs for Teams, because if I leave this in the middle, it would create unnecessary confusions for other users.

[jedieaston]

BTW: the next release of winget will have a way for the user to select what architecture they would like to install, regardless of what architecture they are running for their system (so if you're on a x64 system and want the x86 version of something, --architecture or --a during install will do that): microsoft/winget-cli#1666

As a result, we should start figuring out which of the *.x64 and *.x86 manifests can be merged. Adobe Reader seems like a target, as do the .NET SDKs.

[vedantmgoyal9]

These packages need to moved to Microsoft.dotnetRuntime.5:

• Microsoft.dotnetRuntime.5-x86
• Microsoft.dotnetRuntime.5-x64
• Microsoft.dotnetRuntime.5-arm64

These packages need to moved to Microsoft.dotnetRuntime.3:

• Microsoft.dotnetRuntime.3-x86
• Microsoft.dotnetRuntime.3-x64

These packages need to moved to Microsoft.dotnetRuntime.6:

• Microsoft.dotnetRuntime.6-x64

---

Here are the packages that are still left to be moved: proposed_updates.xlsx

[vedantmgoyal9]

I guess --architecture is now in public preview(?)

[jedieaston]

Yep. Once it's in stable, we can start merging those manifests (unless they are being used as dependencies elsewhere, in which case we'll still have to wait on the schema being extended).

[RokeJulianLockhart]

@jedieaston, you previously mentioned KDE. By "we can start merging manifests", do you mean that if this issue is completed, "#39190 (comment)" shall be possible to remediate? (Unless it is already possible?)

[jedieaston]

The KDE packages aren't separated by architecture, so #39190 doesn't change by it existing. Although I need to see if we can start merging some other manifests, or if we need more support in the client (Adobe Reader, maybe?)