phylum update subcommand is broken

[furi0us333]

Overview

Due to changes in the way release assets are packaged in the current pre-release, the updater fails to find the file it is looking for.

How To Reproduce

Steps to reproduce this behavior:

1. Run phylum update -p
2. See error

Expected Behavior

The updater successfully downloads and installs the correct version.

Additional Context

The new package strategy is nice as you no longer have to download all builds, so this likely requires some fixes in the updater code (https://github.com/phylum-dev/cli/blob/development/cli/src/update.rs). We will probably need to generate minisigs off the platform zips and check those to verify the package.

Current master Assets

Current development Assets

[maxrake]

I think it would also be good to have build target names standardized so as to ensure uniqueness and traceability from the rust build target to the download artifact. For example, instead of a build target like aarch64-apple-darwin becoming phylum-macos-aarch64.zip as a build artifact, it could be more like aarch64-apple-darwin.zip or phylum-aarch64-apple-darwin.zip.

The reference for Rust target triples explains how they are constructed and, interestingly, how they can actually be four parts...and sometimes two!

[furi0us333]

@louislang would you like @kylewillmon to normalize the zip names to Rust build target names as @maxrake suggested?

[louislang]

I don't have super strong opinions. My one qualm against this change is basically <arch>-<company>-<system> which perhaps is less clear? macos is more prominent than darwin, for example.

[maxrake]

Understood. The idea behind this suggestion was to pave the way for making the phylum install be more like the rustup.sh install script...where the script is run on target and determines the correct binary to acquire. Consumers of the phylum CLI tool wouldn't need to know about all the various target triples and which one they need b/c the install script would do that for them (Windows being the exception). Looking at that script, it appears that the sys part of the target triple is determined with a uname -s call.

Since the CLI is built with Rust, the thought was that the Rust targets...and maybe even a good chunk of the rustup script...could be used as is. Even still, mapping from darwin to macos is not a big deal...so long as no future naming conflicts emerge.

[kylewillmon]

If we want phylum update to work from 1.2.0 to 1.3.0, we must include the assets that the already-released 1.2.0 binary expects in the 1.3.0 release.

Or, to put it differently: The easiest way to resolve this issue is to change the assets back to the layout used for the last release.

The layout change was made in #172, but I don't see an explanation there for why the layout was changed. Is anyone opposed to going back to the old layout? (cc: @andreaphylum)

The new layout definitely looks less cluttered. But, as @maxrake points out, we are moving towards a universal install script. And that script will allow people to avoid searching through our release assets.

[andreaphylum]

The layout change was made in #172, but I don't see an explanation there for why the layout was changed. Is anyone opposed to going back to the old layout? (cc: @andreaphylum)

That PR was a general refactoring pass, the main motivation for the switch was indeed just decluttering and simplifying.
In fact, I totally support going towards a single install script, but I'm not sure reverting to the old layout would work well to that end if it costs in tidyness of the artifacts. We would essentially have to offer a worse experience to those users that may need/want to retrieve the release artifacts directly. I haven't looked deeply into this issue and can't offer a good informed solution yet, but here's an idea: we may temporarily put some 1.2.0-to-1.3.0 specific update code until we're fairly sure that most 1.2.0 users have switched over to 1.3.0, or for a fixed amount of time (e.g. a few months), and then just remove that code when we are ready to fully transition to, and support, a new paradigm.

[kylewillmon]

A few points:

• Making upgrade from 1.2.0 work for x86_64 targets (i.e., linux-x86_64 and macos-x86_64) is only possible if we release artifacts with the old layout.
• Current 1.2.0 clients on macos-aarch64 cannot be fixed (because the upgrade code will mistakenly download the x86_64 binary)
• Furthermore, fixing upgrade for macos-x86_64 clients will make macos-aarch64 clients report success, even though they have actually installed the wrong architecture.

Because of this, I do not believe that it is worthwhile to fix phylum update for 1.2.0 clients. Instead, the command will error and exit at the download stage without modifying any installed binaries. So the 2 pieces left of this issue are:

1. Update the docs to explain how to upgrade to 1.3.0 (since phylum update will not work)
2. Redesign the upgrade process for the future so that we can resume use of the phylum update command for 1.3.0 -> 1.3.0+1

[kylewillmon]

Picking this back up today. Here is my plan for a redesign. When phylum update is run, it will:

1. Confirm that the user is on a platform that supports self-update (i.e, aarch64-apple-darwin, x86_64-apple-darwin, or x86_64-unknown-linux).
2. Attempt an escape hatch update (details below)
3. Perform the steps exactly as explained for a fresh install here (i.e., download the zip, verify the signature, unzip it, and run ./install.sh)

Escape hatch update

To ensure that phylum update continues to work in the future even if we change our install process and/or release layout again, I will add an "escape hatch". Here are the steps I propose for the escape hatch:

1. Do an HTTP GET of https://update.phylum.io/update.sh and https://update.phylum.io/update.sh.minisig
2. If there is a 404 error or DNS error, go back to the normal install steps
3. If files are return, verify the signature and run update.sh to perform the update.

[kylewillmon]

To simplify the PRs, I'm splitting off the escape hatch into its own issue, #226

[kylewillmon]

Fixed in #225