Provide a quickstart command to install fcli

[MikeTheSnowman]

Enhancement Request

The TL/DR explanation:

Provide a "quickstart" command that makes it easy to install fcli. We could provide something like:
curl <URL_TO_INSTALL_SCRIPT> | bash

Detailed explanation:

FCLI is currently a standalone binary and we allow for users to determine where they want to "install" the fcli binary.
I believe that there is a large population of users who simply don't want to think about something like that and would like a simpler way to just get the latest version of FCLI.

Even with other popular projects, they provide an installation script that will:

• download the latest version
• add it to directory at a known (documented) location
• possibly add that directory to the PATH environment variable
• possibly update the current user's bashrc file to persist the PATH variable update
• perform additional sanity checks if needed

Adding on to this idea, if we make the installation script available from the git repo, then we can provide a one-liner in the README to make it super simple for users and devops people to download/install the latest version.

For example:

• Publish the installation script to the github repo, or create a gist.
• Update the README with a new Quickstart section that provides the following command curl <URL_TO_INSTALL_SCRIPT> | bash

Inside the installation script, I've found that the following command works well to retrieve the latest version of fcli (for Linux). Please note that this does not make any use of non-standard tools like jq to maintain portability:

curl -s https://api.github.com/repos/fortify/fcli/releases/latest | grep "browser_download_url" | cut -d : -f 2,3 | tr -d \" | grep -Po "https://github.com/fortify/fcli/releases/download/v?[\d\.]+/fcli-linux.tgz$" | xargs curl -OL

For completeness, here's the one-liner for downloading a specific version of fcli:

export FCLI_VERSION="v2.6.0"; curl -OL "https://github.com/fortify/fcli/releases/download/${FCLI_VERSION}/fcli-linux.tgz"

The potential benifits of this would be:

• Easier, quicker, and reproducable installations of fcli.
• Easier for customers to install the latest version of fcli from CI/CD pipelines.
• Ability for us to add in additional installation logic just in case there are additional "migration steps" that need to be done when upgrading from one fcli version to another.

Potential pitfalls:

• An equivilent command for Powershell (Windows) might also need to be created, but we can probably worry about that later.

Examples of other projects offering this method of installation:

• Helm: https://helm.sh/docs/intro/install/#from-script
• K3D: https://k3d.io/v5.7.3/#install-current-latest-release

[rsenden]

Hey @MikeTheSnowman, thanks for the suggestion. This is something that has been on my mind as well, however some potential issues:

• As we're a security company, ideally file integrity should be checked. The install script could check the integrity of the fcli binaries, but we can't automatically verify integrity of the install script itself (unless we provide a very complicated command that includes signature check). Of course, we could just provide similar instructions as the Helm script, stating that you should review the script, and only use the one-liner if you like living on the edge.
• Where do we install fcli, and how to make sure it's on the path? Do we want to use sudo (like in the Helm install script) to install to /usr/local/bin by default? Do we install to ~/fortify/tools/fcli/<version> (which is the default directory for fcli tool fcli install), and how do we then (automatically) get fcli onto the path?
• Ideally, an fcli installation done through the script should be visible in the fcli tool fcli list command, and upgradable through both the script or the installed fcli itself.
• We'd need to do some platform detection to install the proper fcli binary (potentially also catering for installations through bash on Windows).
• Using 'latest' by default is a bad practice; if we ever introduce any breaking changes in fcli (moving to 3.x), existing pipelines that use the install script without explicitly specifying an fcli version might fail.
• Having users specify an explicit version number is safest, but will prevent them from automatically benefiting from bug fixes and new features. Having the script support semantic versions like v2 or v2.3 would make the script extremely complex (and would replicate a lot of functionality that's already available in fcli itself).

Some ideas that come to mind to mitigate some of these issues:

• We could host the script on the v2.x fcli branch and have it read version.txt from the same branch, which should always list the latest v2.x version. If we ever move to a v3.x branch, we would update the script in that branch to read from version.txt on the v3.x branch (having the script on the 2.x branch still read version.txt from the v2.x branch). This way, users can simply run either the v2.x or v3.x script, and get the latest corresponding version, without (much) risk of pipelines breaking due to unexpectedly moving from fcli v2.x to v3.x.
• The install script itself would be maintained on the develop branch; for now it would read version.txt from the v2.x branch but in the future it might read version.txt from the v3.x branch, so users that don't care about backward compatibility can simply run the script from the develop branch.
• We could utilize fcli to bootstrap its own installation (like we do with fortify/github-action/setup). The script could download a specific (or latest v2.x) fcli version, then (possibly) run fcli tool definitions update and fcli tool fcli install to benefit from semantic versioning support and other advanced features like installing wrapper scripts.
• If we utilize a temporary copy of fcli, we could even easily have the script optionally install other Fortify tools. For example, the script could automatically run fcli tool sc-client install ${INSTALL_SC_CLIENT} if that environment variable is defined.

As an additional thought, we could also consider integrating fcli into various package managers. I think a colleague suggested asdf, but we could also consider standard OS package managers like apt. We could also consider having a Node-based installer, with actual installation functionality in a NodeJS library, such that we could easily integrate this into Node-based plugins/integrations like GitHub and ADO.

[MikeTheSnowman]

Hey @rsenden.

Here are my responses to th potential issues that you brought up:

• As we're a security company, ideally file integrity should be checked. The install script could check the integrity of the fcli binaries, but we can't automatically verify integrity of the install script itself (unless we provide a very complicated command that includes signature check). Of course, we could just provide similar instructions as the Helm script, stating that you should review the script, and only use the one-liner if you like living on the edge.

  • [response]: Agreed. Do you think it would be sufficient for us to simply verify the download against the (unsigned) sha256 hash that we provide, or do you think it's a hard requirement that we verify using the signed hash? If we can simply verify againse the unsigned hash, then this can be done easily. After reading your other concerns, I'm now thinking that we can probably offer 3 types of quickstart commands: A command to install the latest dev release (for living on the edge), a command to install the latest version from a specific series of fcli (eg: series v1.x, v2.x, etc.), and a command to install a specific version of fcli.

• Where do we install fcli, and how to make sure it's on the path? Do we want to use sudo (like in the Helm install script) to install to /usr/local/bin by default? Do we install to ~/fortify/tools/fcli/ (which is the default directory for fcli tool fcli install), and how do we then (automatically) get fcli onto the path?

  • [response]: I totally forgot about fcli tool for a moment. But the quickstart script should definitely install fcli in the default location that's recognized/detected by fcli tools. As for persisting fcli onto the PATH env variable, do you see any issues with just append export PATH=~/fortify/tools/fcli/<version>:PATH to ~/.bashrc (for macOS ~.zshrc)? Of course there are other shells out there, but for now I think we can just focus on bash/zsh. Everything described so far should, I hope, avoid the need to use sudo.

• Ideally, an fcli installation done through the script should be visible in the fcli tool fcli list command, and upgradable through both the script or the installed fcli itself.

  • [response]: Agreed!

• We'd need to do some platform detection to install the proper fcli binary (potentially also catering for installations through bash on Windows).

  • [response]: Agreed and this shouldn't be a huge issue. It's just in the case of Windows, we'd need a powershell script written as well.

• Using 'latest' by default is a bad practice; if we ever introduce any breaking changes in fcli (moving to 3.x), existing pipelines that use the install script without explicitly specifying an fcli version might fail.

  • [response]: Understood. I think my response for your first concern has a resonable response to this concern of your.

• Having users specify an explicit version number is safest, but will prevent them from automatically benefiting from bug fixes and new features. Having the script support semantic versions like v2 or v2.3 would make the script extremely complex (and would replicate a lot of functionality that's already available in fcli itself).

  • [response]: Agreed. My response here is the same as above

I'm still thinking about the other ideas that you mentioned as I'm thinking that there's probably a simpler solution. I guess in my head, the develop branch would host the most up to date version of installation script which would maintain backwards aompatability for being able to install any version of fcli (or we simply make the installation script a public gist). I've created an example script (available via gist) that can be invoked in three ways (please note that the script only echos back your "selection"):

1. Install the latest dev release (living on the bleeding edge):

curl -L https://gist.github.com/MikeTheSnowman/88d64142a92374c8835fc2c1c1fe9199/raw/fcli-install-example.sh | bash -s -- dev

2. Install the latest v2.x release:

curl -L https://gist.github.com/MikeTheSnowman/88d64142a92374c8835fc2c1c1fe9199/raw/fcli-install-example.sh | bash -s -- v2.x

3. Install the latest v1.x release:

curl -L https://gist.github.com/MikeTheSnowman/88d64142a92374c8835fc2c1c1fe9199/raw/fcli-install-example.sh | bash -s -- v1.x

4. Install a specific version:

curl -L https://gist.github.com/MikeTheSnowman/88d64142a92374c8835fc2c1c1fe9199/raw/fcli-install-example.sh | bash -s -- v2.2.2

[rsenden]

Hi @MikeTheSnowman, fcli doesn't check (default) installation directories to identify whether a particular tool has been installed or not; it keeps state data about installed tools in the fcli state directory (~/.fortify/fcli/state by default). So, just having the installation script install fcli in the default installation directory won't help to have fcli recognize that installation.

To have fcli recognize the installation, we'd either need to create the appropriate JSON file in the fcli state directory (which would probably be a bad choice as we'd need to duplicate logic/JSON structure across fcli and installation script), or the script must utilize fcli to perform the installation.

So, to utilize installation logic already present in fcli, and have fcli recognize the installation afterwards, I think the installation script would need to:

• Download a fixed fcli version (or possibly latest v2.x by looking at version.txt on the v2.x branch)
• Run fcli tool definitions update using that fixed fcli version
• Run fcli tool fcli install -v <user-supplied version> using that fixed fcli version (defaulting to latest or maybe specific major version like v2)
• Delete the fixed fcli version (or keep it to avoid redownload if installation script is run again)

This is basically what our GitHub Action is currently doing, main drawback is that we're downloading fcli twice (same or different version); once for use by the installation script, once through the fcli tool fcli install command.

As for verifying integrity of the installation script, not sure whether we can create a one-line command that that downloads the script, verifies its integrity, and then runs it (independent of whether we verify hash or signature). I think signature is better than hash. The hash will change whenever we update the script, so we'd need to constantly update the documentation that shows the one-line command to use the new hash. Verifying signature uses a constant public key, so we don't need to update the one-line command on every script change.

As for sudo, maybe we should support both with and without sudo, depending on whether the users wants to install fcli for him/herself only, or globally for all users. For a single user installation, we would default to ~/fortify/tools/fcli/<version>/bin, for a global installation we could default to /opt/fortify/fcli/<version>/bin

As for PATH, not sure whether we should touch user's .bashrc or similar, maybe you could check how other tools are doing this (other than installing to /usr/local/bin like the Helm script). It also depends on whether you want to do a single-user or global install; for the latter you may want to add the PATH to some file in /etc.

Note that fcli also installs wrapper scripts, for example a command like fcli tool fcli install --base-dir /opt/fortify would install the fcli executable to /opt/fortify/fcli/<version>/bin/fcli and create a wrapper script /opt/fortify/bin/fcli. Only the latter needs to be on the PATH and will be automatically updated if you later install a more recent fcli version.

Potentially we could have fcli support installing the wrapper scripts to a different directory like /usr/local/bin or ~/bin (both of which are usually already on the PATH), and/or we could have the installation script create symbolic links from /usr/local/bin/fcli to those wrapper scripts.