Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added gitlab support, simplified version sorting with ls-remote #58

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+59 −17
Open

feat: added gitlab support, simplified version sorting with ls-remote #58

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b36b7a6
feat: added gitlab support, simplified version sorting with ls-remote
stephanGarland Jan 15, 2023
2b483fd
fix: adds a regex requiring that the tag match some vague form of sem…
stephanGarland Jan 15, 2023
3d909be
fix: adds a fallback check for user's git ls-remote not supporting sort
stephanGarland Jan 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion template/bin/list-all
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ plugin_dir=$(dirname "$(dirname "$current_script_path")")
# shellcheck source=../lib/utils.bash
source "${plugin_dir}/lib/utils.bash"

list_all_versions | sort_versions | xargs echo
list_all_versions | xargs echo
74 changes: 58 additions & 16 deletions template/lib/utils.bash
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,21 @@

set -euo pipefail

# TODO: Ensure this is the correct GitHub homepage where releases can be downloaded for <YOUR TOOL>.
GH_REPO="<TOOL REPO>"
GIT_REMINDER_FILE="${ASDF_INSTALL_PATH}/.upgradegit"
GIT_VERSION=$(git --version)
GIT_VERSION=${GIT_VERSION##* }
# TODO: Ensure this is the correct GitHub/GitLab homepage where releases can be downloaded for <YOUR TOOL>.
REPO="<TOOL REPO>"
TOOL_NAME="<YOUR TOOL>"
TOOL_TEST="<TOOL CHECK>"
IS_GITHUB=$(
[[ "$REPO" =~ "github" ]]
echo $?
)

git_supports_sort() {
awk '{ split($0,a,"."); if ((a[1] < 2) || (a[2] < 18)) { print "1" } else { print "0" } }' <<<"$1"
}

fail() {
echo -e "asdf-$TOOL_NAME: $*"
Expand All @@ -14,26 +25,53 @@ fail() {

curl_opts=(-fsSL)

# NOTE: You might want to remove this if <YOUR TOOL> is not hosted on GitHub releases.
if [ $(git_supports_sort "${GIT_VERSION}") -eq 0 ]; then
rm -f "${GIT_REMINDER_FILE}"
GIT_SUPPORTS_SORT=0
else
GIT_SUPPORTS_SORT=1
if [ ! -f "${GIT_REMINDER_FILE}" ]; then
printf "consider upgrading git to a version >= 2.18.0 for faster asdf - you have v${GIT_VERSION}\n"
touch "${GIT_REMINDER_FILE}"
fi
fi

# NOTE: You might want to remove this if <YOUR TOOL> is not hosted on GitHub or GitLab releases.
if [ -n "${GITHUB_API_TOKEN:-}" ]; then
curl_opts=("${curl_opts[@]}" -H "Authorization: token $GITHUB_API_TOKEN")
elif [ -n "${GITLAB_API_TOKEN:-}" ]; then
curl_opts=("${curl_opts[@]}" -H "Authorization: token $GITLAB_API_TOKEN")
fi

sort_versions() {
sed 'h; s/[+-]/./g; s/.p\([[:digit:]]\)/.z\1/; s/$/.z/; G; s/\n/ /' |
LC_ALL=C sort -t. -k 1,1 -k 2,2n -k 3,3n -k 4,4n -k 5,5n | awk '{print $2}'
}

list_github_tags() {
git ls-remote --tags --refs "$GH_REPO" |
grep -o 'refs/tags/.*' | cut -d/ -f3- |
sed 's/^v//' # NOTE: You might want to adapt this sed to remove non-version strings from tags
# NOTE: If <YOUR TOOL> doesn't issue releases according to something resembling semver,
# you will need to edit the awk regex.
list_remote_tags() {
if [ ${GIT_SUPPORTS_SORT} -eq 0 ]; then
git -c 'versionsort.suffix=a' -c 'versionsort.suffix=b' \
-c 'versionsort.suffix=r' -c 'versionsort.suffix=p' \
-c 'versionsort.suffix=-' -c 'versionsort.suffix=_' \
ls-remote --exit-code --tags --refs --sort="version:refname" "$REPO" |
awk -F'[/v]' '$NF ~ /^[0-9]+.*/ { print $NF }' || fail "no releases found"
else
git ls-remote --exit-code --tags --refs "$REPO" |
awk -F'[/v]' '$NF ~ /^[0-9]+.*/ { print $NF }' || fail "no releases found"
fi
}

list_all_versions() {
# TODO: Adapt this. By default we simply list the tag names from GitHub releases.
# Change this function if <YOUR TOOL> has other means of determining installable versions.
list_github_tags
if [ ${GIT_SUPPORTS_SORT} -eq 0 ]; then
list_remote_tags
else
list_remote_tags | sort_versions
fi
}

# NOTE: This is a fallback for if the user's installed version of git doesn't support sorting.
sort_versions() {
sed 'h; s/[+-]/./g; s/.p\([[:digit:]]\)/.z\1/; s/$/.z/; G; s/\n/ /' |
LC_ALL=C sort -t. -k 1,1 -k 2,2n -k 3,3n -k 4,4n -k 5,5n | awk '{print $2}'
}

download_release() {
Expand All @@ -42,10 +80,14 @@ download_release() {
filename="$2"

# TODO: Adapt the release URL convention for <YOUR TOOL>
url="$GH_REPO/archive/v${version}.tar.gz"

if [ $IS_GITHUB -eq 0 ]; then
url="$REPO/archive/v${version}.tar.gz"
Copy link
Collaborator

@jthegedus jthegedus Jan 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is GitHub it should be $REPO/releases/

Copy link
Author

@stephanGarland stephanGarland Jan 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've found an interesting difference in these endpoints, now that I look at it:

# using ccache/ccache as example repo
❯ curl -LI -w "code: %{response_code}\ncontent-type: %{content_type}\neffective_url: %{url_effective}\ntime_total: %{time_total}\n" https://github.com/ccache/ccache/archive/v4.7.4.tar.gz
...
code: 200
content-type: application/x-gzip
effective_url: https://codeload.github.com/ccache/ccache/tar.gz/refs/tags/v4.7.3
time_total: 0.158385

❯ curl -LI -w "code: %{response_code}\ncontent-type: %{content_type}\neffective_url: %{url_effective}\ntime_total: %{time_total}\n" https://github.com/ccache/ccache/releases/download/v4.7.4/ccache-4.7.4.tar.gz
...
code: 200
content-type: application/octet-stream
effective_url: https://objects.githubusercontent.com/github-production-release-asset-2e65be/461102/81edd037-87b4-4b75-ad06-9325550c3815?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230122%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230122T152917Z&X-Amz-Expires=300&X-Amz-Signature=fe99692a02cfdca69367c08a6205aec3a74b2736d45e4cc55bd7bd450547c9ae&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=461102&response-content-disposition=attachment%3B%20filename%3Dccache-4.7.4.tar.gz&response-content-type=application%2Foctet-stream
time_total: 0.366820

The former appears to be a direct access link, and the latter is going to S3 (and has Varnish in front of it, as it also has a via: 1.1 Varnish response header; it was also a cache miss). Both include a redirect.

Both include x-content-type-options: nosniff and content-disposition: attachment; filename=ccache-4.7.4.tar.gz, but the former is explicitly declaring the type to be gzip, and the latter directs the browser to figure it out.

Either way, I don't care and can adjust if needed, but wanted to clarify that both are valid.

else
url="$REPO/-/archive/${version}/${TOOL_NAME}-${version}.tar.gz"
fi
printf "%s: %s\n" "url" "$url"
echo "* Downloading $TOOL_NAME release $version..."
curl "${curl_opts[@]}" -o "$filename" -C - "$url" || fail "Could not download $url"
curl "${curl_opts[@]}" -o "$filename" "$url" || fail "Could not download $url"
}

install_version() {
Expand Down