Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gh-action] Refactored the "Add new docker versions" workflow #142

Merged
merged 2 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/add_new_versions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
- name: Check if new versions available
id: check-versions
run: |
python workflow_scripts/check-for-new-versions.py
python -u workflow_scripts/check-for-new-versions.py
env:
EXCLUDED_VERSIONS: "v20.10.x,v23.0.x"

Expand All @@ -54,7 +54,7 @@ jobs:
env:
NEW_VERSIONS: ${{ env.NEW_VERSIONS }}
run: |
python workflow_scripts/gen-new-version-files.py
python -u workflow_scripts/gen-new-version-files.py

- name: Create branch, commit and push
if: ${{ env.pr_exist == 'false' && env.PR_TITLE != '' }}
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ TEST_OS_IMAGE_TAG[2]="10"

On each merge to master, scripts in `dist/` will be uploaded to `install-docker-dev` bucket and can be retrieved using https://releases.rancher.com/install-docker-dev/$VERSION.sh. The commit of the version that was uploaded can be found on https://releases.rancher.com/install-docker-dev/VERSION

When testing has been completed, a tag can be created to upload the scripts in `dist/` to `install-docker` (https://releases.rancher.com/install-docker/$VERSION.sh). A tag consists of the latest Docker version in the repository (for example, `20.10.12`) and epoch timestamp (in case we need to release same set of versions with changes). The tag can be generated using `scripts/generate-release-tag`, or you can use the GitHub Actions workflow [Create release tag](https://github.com/rancher/install-docker/actions/workflows/create-tag.yml) directly. The tag of the version that was uploaded can be found on https://releases.rancher.com/install-docker/VERSION
When testing has been completed, the release job needs to be run to upload the scripts in `dist/` to `install-docker` (https://releases.rancher.com/install-docker/$VERSION.sh). A tag consists of the latest Docker version in the repository (for example, `20.10.12`) and epoch timestamp (in case we need to release same set of versions with changes). The release job will automatically create a tag and release the artifacts to the specified upload bucket. To initiate the release process, use the GitHub Actions workflow [Creates a tag and upload release](https://github.com/rancher/install-docker/actions/workflows/create-release.yml). You can find the tag of the version that was uploaded at https://releases.rancher.com/install-docker/VERSION

## Previous manual instructions to add a new version

Expand Down
1 change: 0 additions & 1 deletion workflow_scripts/check-for-new-versions.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import os
import subprocess
import requests

# Constants
Expand Down
55 changes: 13 additions & 42 deletions workflow_scripts/gen-new-version-files.py
Original file line number Diff line number Diff line change
@@ -1,48 +1,29 @@
import os
import subprocess
import requests
import semver

# Constants
DIST_FOLDER = './dist'
NEW_VERSIONS = os.environ.get("NEW_VERSIONS","")
SCRIPT_EXTENSION=".sh"

if NEW_VERSIONS == "":
print("no new versions available, NEW_VERSIONS env variable is empty")
exit(1)

def get_max_version(ver1, ver2):
if ver1.startswith('v'):
ver1 = ver1[1:]
if ver2.startswith('v'):
ver2 = ver2[1:]

ver1_tuple = tuple(map(int, ver1.split('.')))
ver2_tuple = tuple(map(int, ver2.split('.')))
if ver1 > ver2:
return ver1
return ver2

def format_version(v):
if v.startswith('v'):
return v[1:]
return v

def get_last_added_version(files_dir):
max_modification_time = 0.0
last_added_version = ""
last_added_version = "0.0.0"
for file in os.listdir(files_dir):
if file.endswith('.sh') and file.count('.') == 3:
if file.endswith(SCRIPT_EXTENSION) and file.count('.') == 3:
file_path = os.path.join(files_dir, file)
modification_time = os.path.getmtime(file_path)
file_version = 'v' + file[:-3]
file_version = file.removesuffix(SCRIPT_EXTENSION)
if modification_time > max_modification_time:
max_modification_time = modification_time
last_added_version = file_version
elif modification_time == max_modification_time:
if last_added_version == "":
last_added_version = file_version
else:
last_added_version = get_max_version(last_added_version,file_version)
last_added_version = semver.max_ver(last_added_version,file_version)

return last_added_version

Expand All @@ -59,16 +40,9 @@ def get_version_dict(versions):
version_dict = {}

for version in versions:
version_parts = version.split('.')
major_minor = version_parts[0] + '.' + version_parts[1]

if major_minor in version_dict:
current_version = tuple(map(int, version_parts[2]))
max_version = tuple(map(int, version_dict[major_minor].split('.')[2]))
if current_version > max_version:
version_dict[major_minor] = version
else:
version_dict[major_minor] = version
version_parts = semver.parse(version)
major_minor = f"{version_parts['major']}.{version_parts['minor']}"
version_dict[major_minor] = semver.max_ver(version_dict.setdefault(major_minor, version), version)

return version_dict

Expand All @@ -77,19 +51,16 @@ def main():
last_added_version = get_last_added_version(DIST_FOLDER)
print("Last added version:",last_added_version)

new_versions = NEW_VERSIONS.split(',')
formatted_new_versions = list(map(format_version,new_versions))
print("Formatted new versions: ", formatted_new_versions)
version_list = [version.removeprefix('v') for version in NEW_VERSIONS.split(',')]
print("Formatted new versions: ", version_list)

for version in formatted_new_versions:
for version in version_list:
generate_diffs(last_added_version, version)

versions_string = ",".join(new_versions)

print("running generate script")
subprocess.run(["bash", "./scripts/generate"], check=True)

version_dict = get_version_dict(formatted_new_versions)
version_dict = get_version_dict(version_list)
print("version dictionary for symlink: ",version_dict)

for major_minor,version in version_dict.items():
Expand Down
1 change: 1 addition & 0 deletions workflow_scripts/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
Requests==2.31.0
semver==3.0.2