Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding test-mesa Github Action workflow #683

Merged
merged 56 commits into from
Sep 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
713fedf
adding github action to install mesa
pmocz Jul 18, 2024
e6096cd
rename
pmocz Jul 18, 2024
ac0e5c1
switch checkout
pmocz Jul 18, 2024
7a174eb
adding shell
pmocz Jul 18, 2024
a037d3b
try add a test-mesa
pmocz Jul 18, 2024
dd07e38
progress on yaml file
pmocz Jul 18, 2024
e40fc14
[ci skip] partial progress
pmocz Jul 18, 2024
3fcf1c5
yml
pmocz Jul 18, 2024
8533ea7
change checkout
pmocz Jul 18, 2024
35f3c3d
install mesa test
pmocz Jul 18, 2024
16c9fb4
add more info
pmocz Jul 18, 2024
9dbd65c
sudo
pmocz Jul 18, 2024
7f23fbe
env vars
pmocz Jul 18, 2024
d2e6ce5
paths
pmocz Jul 18, 2024
9287458
mkdir
pmocz Jul 18, 2024
f228a15
paths
pmocz Jul 18, 2024
47823e5
typo
pmocz Jul 18, 2024
f0009b2
paths
pmocz Jul 18, 2024
8ca41f8
more paths
pmocz Jul 18, 2024
5b4f49e
paths
pmocz Jul 18, 2024
fa46223
paths
pmocz Jul 18, 2024
cb1b7eb
env
pmocz Jul 18, 2024
219d741
simplifiy
pmocz Jul 18, 2024
6e29c08
fix shell str compare
pmocz Jul 18, 2024
43c3e6f
typo
pmocz Jul 18, 2024
2a64f4c
str cmp
pmocz Jul 18, 2024
0ace945
space
pmocz Jul 18, 2024
680709f
fix echo
pmocz Jul 19, 2024
58b3ac7
test
pmocz Jul 19, 2024
07df484
echo
pmocz Jul 19, 2024
8e029b0
test
pmocz Jul 19, 2024
612fea0
test
pmocz Jul 19, 2024
ed1a7c6
echo
pmocz Jul 19, 2024
6b755d4
echo
pmocz Jul 19, 2024
497b89e
echo
pmocz Jul 19, 2024
43af9df
python
pmocz Jul 19, 2024
32ead6e
temp disable
pmocz Jul 19, 2024
fab989b
test
pmocz Jul 19, 2024
8b0370a
test
pmocz Jul 19, 2024
222736f
test
pmocz Jul 19, 2024
8afb4d2
env
pmocz Jul 19, 2024
73d1a98
test
pmocz Jul 19, 2024
c298081
test2
pmocz Jul 19, 2024
0843fc8
test2
pmocz Jul 19, 2024
7b5820a
try again
pmocz Jul 19, 2024
b4acc34
enable
pmocz Jul 19, 2024
eec1abd
test
pmocz Jul 19, 2024
dbb7f77
print git hashes
pmocz Jul 19, 2024
116c773
more info
pmocz Jul 19, 2024
13eb682
try ref
pmocz Jul 19, 2024
a2de335
updating auto-tests
pmocz Jul 19, 2024
74dee6d
removing automatic testing on every PR for now
pmocz Jul 19, 2024
630b74b
Merge remote-tracking branch 'origin/main' into feature/pmocz/actions
pmocz Aug 21, 2024
b53a3db
adding quick testing on PRs
pmocz Aug 21, 2024
2e0169d
add new SDK version
pmocz Sep 4, 2024
4473d7b
adding sdk 24.7.1 url
pmocz Sep 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions .github/actions/install-mesa/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name : 'install-mesa'

description: 'Download and install the MESA SDK and install MESA'

inputs:
sdk:
description: 'The version of the MESA SDK to install'
required: false
default: '23.7.3'

runs:

using: 'composite'

steps:
- name: Delete unused packages
run: |
# runners have 150GB of disk space and the Ubuntu image is big so we sometimes found it ran out of space for MESA
# others have encountered this too so we took commands from this action
# https://github.com/jlumbroso/free-disk-space
# currently commented because we don't need it and these commands can need changing when the Ubuntu image changes
sudo rm -rf /usr/local/lib/android
# sudo apt-get remove -y '^aspnetcore-.*'
# sudo apt-get remove -y '^dotnet-.*' # 990 MB
# sudo apt-get remove -y '^llvm-.*' # 1052 MB
# sudo apt-get remove -y 'php.*' # 56.6 MB
# sudo apt-get remove -y '^mysql-.*' # 209 MB
# sudo apt-get remove -y azure-cli google-cloud-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri # 2274 MB
sudo apt-get autoremove -y # 771 MB
sudo apt-get clean
shell: bash

- name: Create LFS file list
run: |
git lfs ls-files -l | cut -d' ' -f1 | sort > .lfs-assets-id
shell: bash

- name: Restore LFS cache
uses: actions/cache@v4
id: lfs-cache
with:
path: .git/lfs
key: ${{ runner.os }}-${{ hashFiles('.lfs-assets-id') }}-v1

- name: Git LFS Pull
run: git lfs pull
shell: bash
if: steps.lfs-cache.outputs.cache-hit != 'true'

- name: Git LFS Checkout
run: git lfs checkout
shell: bash
if: steps.lfs-cache.outputs.cache-hit == 'true'

- name: Install dependencies Linux
run: |
sudo apt-get -y update
sudo apt-get -y install wget binutils make perl libx11-6 libx11-dev zlib1g zlib1g-dev tcsh
shell: bash

- uses: actions/cache@v4
id: cache
with:
path: |
mesasdk-x86_64-linux-${{inputs.sdk}}.tar.gz
key: ${{ runner.os }}-${{inputs.sdk}}

- name: Get SDK ${{ runner.os }} '21.4.1'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( inputs.sdk == '21.4.1') }}
run: |
wget -q https://zenodo.org/record/5802444/files/mesasdk-x86_64-linux-21.4.1.tar.gz
shell: bash

- name: Get SDK ${{ runner.os }} '22.6.1'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( inputs.sdk == '22.6.1') }}
run: |
wget -q https://zenodo.org/record/7457681/files/mesasdk-x86_64-linux-22.6.1.tar.gz
shell: bash

- name: Get SDK ${{ runner.os }} '23.7.3'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( inputs.sdk == '23.7.3') }}
run: |
wget -q https://zenodo.org/record/10624843/files/mesasdk-x86_64-linux-23.7.3.tar.gz
shell: bash

- name: Get SDK ${{ runner.os }} '24.7.1'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( inputs.sdk == '24.7.1') }}
run: |
wget -q https://zenodo.org/records/13768913/files/mesasdk-x86_64-linux-24.7.1.tar.gz
shell: bash

- name: Unpack SDK ${{ runner.os }} ${{inputs.sdk}}
run: |
tar xvf mesasdk-x86_64-linux-${{inputs.sdk}}.tar.gz
shell: bash

- name: Compile
run: |
# Linux runners have 4 cores
# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
export OMP_NUM_THREADS=4
export NPROCS=4
export "MESASDK_ROOT=$(readlink -f mesasdk)"
source "${MESASDK_ROOT}/bin/mesasdk_init.sh"
export "MESA_DIR=$(readlink -f ./)"
# Save environment variables to Github environment
echo "OMP_NUM_THREADS=${OMP_NUM_THREADS}" >> $GITHUB_ENV
echo "NPROCS=${NPROCS}" >> $GITHUB_ENV
echo "MESASDK_ROOT=${MESASDK_ROOT}" >> $GITHUB_ENV
echo "MESA_DIR=${MESA_DIR}" >> $GITHUB_ENV
# Everything is run as root so we need to disable the root check in the install script
sed -i 's/\${EUID:-\$(id -u)}/1/' install
# Turn off caching during build to save more space
sed -i 's/use_cache_for_eos = .true./use_cache_for_eos = .false./g' $MESA_DIR/eos/public/eos_def.f90
sed -i 's/use_cache = .true./use_cache = .false./g' $MESA_DIR/star/private/star_private_def.f90
./install
if [ ! -f lib/libbinary.a ]; then
exit 1
fi
shell: bash
97 changes: 5 additions & 92 deletions .github/workflows/build-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,102 +7,15 @@ jobs:
strategy:
fail-fast: false
matrix:
sdk: ["21.4.1", "22.6.1", "23.7.3"]
sdk: ["22.6.1", "23.7.3", "24.7.1"]

runs-on: ubuntu-latest

steps:
- name: Delete unused packages
run: |
# runners have 150GB of disk space and the Ubuntu image is big so we sometimes found it ran out of space for MESA
# others have encountered this too so we took commands from this action
# https://github.com/jlumbroso/free-disk-space
# currently commented because we don't need it and these commands can need changing when the Ubuntu image changes
sudo rm -rf /usr/local/lib/android
# sudo apt-get remove -y '^aspnetcore-.*'
# sudo apt-get remove -y '^dotnet-.*' # 990 MB
# sudo apt-get remove -y '^llvm-.*' # 1052 MB
# sudo apt-get remove -y 'php.*' # 56.6 MB
# sudo apt-get remove -y '^mysql-.*' # 209 MB
# sudo apt-get remove -y azure-cli google-cloud-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri # 2274 MB
sudo apt-get autoremove -y # 771 MB
sudo apt-get clean

- name: Checkout
uses: actions/[email protected]

- name: Create LFS file list
run: |
git lfs ls-files -l | cut -d' ' -f1 | sort > .lfs-assets-id

- name: Restore LFS cache
uses: actions/cache@v3
id: lfs-cache
with:
path: .git/lfs
key: ${{ runner.os }}-${{ hashFiles('.lfs-assets-id') }}-v1

- name: Git LFS Pull
run: git lfs pull
if: steps.lfs-cache.outputs.cache-hit != 'true'

- name: Git LFS Checkout
run: git lfs checkout
if: steps.lfs-cache.outputs.cache-hit == 'true'

- name: Install dependencies Linux
run: |
sudo apt-get -y update
sudo apt-get -y install wget binutils make perl libx11-6 libx11-dev zlib1g zlib1g-dev tcsh
shell: bash
uses: actions/checkout@v4

- uses: actions/cache@v3
id: cache
- name: Install MESA on ${{ runner.os }} with SDK ${{matrix.sdk}}
uses: ./.github/actions/install-mesa
with:
path: |
mesasdk-x86_64-linux-${{matrix.sdk}}.tar.gz
key: ${{ runner.os }}-${{matrix.sdk}}


- name: Get SDK ${{ runner.os }} '21.4.1'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( matrix.sdk == '21.4.1') }}
run: |
wget -q https://zenodo.org/record/5802444/files/mesasdk-x86_64-linux-21.4.1.tar.gz
shell: bash

- name: Get SDK ${{ runner.os }} '22.6.1'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( matrix.sdk == '22.6.1') }}
run: |
wget -q https://zenodo.org/record/7457681/files/mesasdk-x86_64-linux-22.6.1.tar.gz
shell: bash

- name: Get SDK ${{ runner.os }} '23.7.3'
if: ${{ (steps.cache.outputs.cache-hit != 'true') && ( matrix.sdk == '23.7.3') }}
run: |
wget -q https://zenodo.org/record/10624843/files/mesasdk-x86_64-linux-23.7.3.tar.gz
shell: bash

- name: Unpack SDK ${{ runner.os }} ${{matrix.sdk}}
run: |
tar xvf mesasdk-x86_64-linux-${{matrix.sdk}}.tar.gz
shell: bash

- name: Compile
shell: bash
run: |
# Linux runners have 4 cores
# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
export OMP_NUM_THREADS=4
export NPROCS=4
export "MESASDK_ROOT=$(readlink -f mesasdk)"
source "${MESASDK_ROOT}/bin/mesasdk_init.sh"
export "MESA_DIR=$(readlink -f ./)"
# Everything is run as root so we need to disable the root check in the install script
sed -i 's/\${EUID:-\$(id -u)}/1/' install
# Turn off caching during build to save more space
sed -i 's/use_cache_for_eos = .true./use_cache_for_eos = .false./g' $MESA_DIR/eos/public/eos_def.f90
sed -i 's/use_cache = .true./use_cache = .false./g' $MESA_DIR/star/private/star_private_def.f90
./install
if [ ! -f lib/libbinary.a ]; then
exit 1
fi
sdk: ${{matrix.sdk}}
108 changes: 108 additions & 0 deletions .github/workflows/test-mesa.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
name: Test Mesa

on: [workflow_dispatch, release]

jobs:
test_mesa:
environment: gh-action-testhub
strategy:
fail-fast: false

runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.ref }}

- name: Print git hash
run: |
cd $GITHUB_WORKSPACE
pwd
git branch
git log --pretty=format:'%h' -n 1
shell: bash

- name: Install MESA on ${{ runner.os }} with SDK 24.7.1
uses: ./.github/actions/install-mesa
with:
sdk: "24.7.1"

- name: Checkout mesa_test
uses: actions/checkout@v4
with:
repository: MESAHub/mesa_test
path: mesa_test # Store in $GITHUB_WORKSPACE/mesa_test

- name: Install mesa_test
run: |
cd $GITHUB_WORKSPACE/mesa_test
sudo gem install mesa_test
cd $GITHUB_WORKSPACE
mkdir /home/runner/.mesa_test
shell: bash

- name: Create mesa_test config file
env:
OMP_NUM_THREADS: ${{env.OMP_NUM_THREADS}}
NPROCS: ${{env.NPROCS}}
MESASDK_ROOT: ${{env.MESASDK_ROOT}}
MESA_DIR: ${{env.MESA_DIR}}
run: |
MESA_TEST_YML=/home/runner/.mesa_test/config.yml
echo "---" > $MESA_TEST_YML
echo "computer_name: GitHub_Runner" >> $MESA_TEST_YML
echo "email: ${{ secrets.MESA_TEST_EMAIL }}" >> $MESA_TEST_YML
echo "password: ${{ secrets.MESA_TEST_PASSWORD }}" >> $MESA_TEST_YML
echo "logs_token: ${{ secrets.MESA_TEST_LOGS_TOKEN }}" >> $MESA_TEST_YML
echo "github_protocol: :ssh" >> $MESA_TEST_YML
echo "mesa_mirror: $GITHUB_WORKSPACE/mirror" >> $MESA_TEST_YML
echo "mesa_work: $GITHUB_WORKSPACE" >> $MESA_TEST_YML
echo "platform: Linux" >> $MESA_TEST_YML
echo "platform_version: Ubuntu" >> $MESA_TEST_YML
shell: bash

- name: Test Problem 13
env:
OMP_NUM_THREADS: ${{env.OMP_NUM_THREADS}}
NPROCS: ${{env.NPROCS}}
MESASDK_ROOT: ${{env.MESASDK_ROOT}}
MESA_DIR: ${{env.MESA_DIR}}
run: |
source "${MESASDK_ROOT}/bin/mesasdk_init.sh"
mesa_test test 13
shell: bash

- name: Test Problem 15
env:
OMP_NUM_THREADS: ${{env.OMP_NUM_THREADS}}
NPROCS: ${{env.NPROCS}}
MESASDK_ROOT: ${{env.MESASDK_ROOT}}
MESA_DIR: ${{env.MESA_DIR}}
run: |
source "${MESASDK_ROOT}/bin/mesasdk_init.sh"
mesa_test test 15
shell: bash

- name: Test Problem 29
env:
OMP_NUM_THREADS: ${{env.OMP_NUM_THREADS}}
NPROCS: ${{env.NPROCS}}
MESASDK_ROOT: ${{env.MESASDK_ROOT}}
MESA_DIR: ${{env.MESA_DIR}}
run: |
source "${MESASDK_ROOT}/bin/mesasdk_init.sh"
mesa_test test 29
shell: bash

- name: Test Problem 41
env:
OMP_NUM_THREADS: ${{env.OMP_NUM_THREADS}}
NPROCS: ${{env.NPROCS}}
MESASDK_ROOT: ${{env.MESASDK_ROOT}}
MESA_DIR: ${{env.MESA_DIR}}
run: |
source "${MESASDK_ROOT}/bin/mesasdk_init.sh"
mesa_test test 41
shell: bash