Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cb refactoring rebase #597

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
a9a23b9
Bump sqlparse from 0.4.4 to 0.5.0 (#558)
dependabot[bot] Apr 22, 2024
36eb46f
Bump tqdm from 4.66.1 to 4.66.3 (#569)
dependabot[bot] May 4, 2024
fa7f0f1
Bump werkzeug from 3.0.1 to 3.0.3 (#570)
dependabot[bot] May 7, 2024
a05fcd5
Bump jinja2 from 3.1.3 to 3.1.4 (#571)
dependabot[bot] May 7, 2024
b2fc3e6
Bump mlflow from 2.10.1 to 2.12.1 (#575)
dependabot[bot] May 17, 2024
495d5b9
Bump gunicorn from 21.2.0 to 22.0.0 (#576)
dependabot[bot] May 17, 2024
bdd102a
Bump requests from 2.31.0 to 2.32.0 (#578)
dependabot[bot] May 21, 2024
beccd4c
[CI] Run tests through GitHub Actions (#573)
NicolasGensollen May 22, 2024
2861e9d
[CI] Skip tests when PR is in draft mode (#592)
NicolasGensollen May 23, 2024
f5de251
[CI] Test train workflow on GPU machine (#590)
NicolasGensollen May 23, 2024
69b3538
[CI] Port remaining GPU tests to GitHub Actions (#593)
NicolasGensollen May 23, 2024
c9d9252
[CI] Remove GPU pipeline from Jenkinsfile (#594)
NicolasGensollen May 24, 2024
753f04e
[CI] Port remaining non GPU tests to GitHub Actions (#581)
NicolasGensollen May 24, 2024
c424d77
[CI] Remove jenkins related things (#595)
NicolasGensollen May 24, 2024
36c7f47
add pydantic to dependency (#556)
camillebrianceau Apr 16, 2024
42eaaab
Creation of a PredictManager from MapsManager (#557)
camillebrianceau Apr 16, 2024
f48c443
Creation of the trainer (#559)
thibaultdvx Apr 16, 2024
09669ea
Trainer config (#561)
thibaultdvx Apr 24, 2024
f6ea938
Creation of GenerateConfig (#563)
camillebrianceau Apr 25, 2024
dc7589f
Add a DataClass for predict pipeline (#560)
camillebrianceau Apr 25, 2024
4f81916
Put train pipeline inside CLI files (#565)
thibaultdvx Apr 25, 2024
2be01ec
Reorganize train folder (#566)
thibaultdvx Apr 26, 2024
2dd93a8
Cleaning (#564)
camillebrianceau Apr 29, 2024
49d1659
Clean train cli (#567)
thibaultdvx Apr 30, 2024
912f4bd
Creation of Prepare Data Config (#568)
camillebrianceau May 15, 2024
3d0a967
add config folder (#580)
camillebrianceau May 22, 2024
7821a1a
Add enum (#589)
camillebrianceau May 23, 2024
57631a3
Config class in trainer (#574)
thibaultdvx May 23, 2024
ff2c650
Trainer adaptation to data class (#588)
camillebrianceau May 29, 2024
ceafbe5
Merge branch 'refactoring' into cb_refactoring_rebase
camillebrianceau May 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/test_cli.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: CLI Tests

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-cli:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- cpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run CLI tests
run: |
make env.conda
source /builds/miniconda3/etc/profile.d/conda.sh
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_cli_report.xml \
--disable-warnings \
--verbose \
test_cli.py
53 changes: 53 additions & 0 deletions .github/workflows/test_generate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Generate Tests

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-generate:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- cpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run tests for generate task
run: |
make env.conda
source /builds/miniconda3/etc/profile.d/conda.sh
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_generate_report.xml \
--disable-warnings \
--verbose \
--basetemp=$HOME/tmp/generate \
--input_data_directory=/mnt/data/data_ci \
test_generate.py
- name: Cleaning
run: |
rm -rf $HOME/tmp/generate
53 changes: 53 additions & 0 deletions .github/workflows/test_interpret.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Interpretation Tests (GPU)

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-interpret-gpu:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- gpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run tests for Interpret task on GPU
run: |
make env.conda
source "${HOME}/miniconda3/etc/profile.d/conda.sh"
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_interpret_report.xml \
--disable-warnings \
--verbose \
--basetemp=$HOME/actions_runner_workdir/interpret \
--input_data_directory=/mnt/data/clinicadl_data_ci/data_ci \
test_interpret.py
- name: Cleaning
run: |
rm -rf $HOME/actions_runner_workdir/interpret/*
53 changes: 53 additions & 0 deletions .github/workflows/test_predict.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Predict Tests

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-predict:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- cpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run tests for predict task
run: |
make env.conda
source /builds/miniconda3/etc/profile.d/conda.sh
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_predict_report.xml \
--disable-warnings \
--verbose \
--basetemp=$HOME/tmp/predict \
--input_data_directory=/mnt/data/data_ci \
test_predict.py
- name: Cleaning
run: |
rm -rf $HOME/tmp/predict/*
53 changes: 53 additions & 0 deletions .github/workflows/test_prepare_data.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Prepare data Tests

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-prepare-data:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- cpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run tests for prepare data task
run: |
make env.conda
source /builds/miniconda3/etc/profile.d/conda.sh
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_prepare_data_report.xml \
--disable-warnings \
--verbose \
--basetemp=$HOME/tmp/prepare_data \
--input_data_directory=/mnt/data/data_ci \
test_prepare_data.py
- name: Cleaning
run: |
rm -rf $HOME/tmp/prepare_data/*
53 changes: 53 additions & 0 deletions .github/workflows/test_quality_checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Quality Check Tests

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-quality-check:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- cpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run tests for Quality Check
run: |
make env.conda
source /builds/miniconda3/etc/profile.d/conda.sh
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_quality_check_report.xml \
--disable-warnings \
--verbose \
--basetemp=$HOME/tmp/quality_checks \
--input_data_directory=/mnt/data/data_ci \
test_qc.py
- name: Cleaning
run: |
rm -rf $HOME/tmp/quality_checks/*
53 changes: 53 additions & 0 deletions .github/workflows/test_random_search.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Random Search Tests (GPU)

on:
push:
branches: ["dev", "refactoring"]
pull_request:
branches: ["dev", "refactoring"]

permissions:
contents: read

concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true

env:
POETRY_VERSION: '1.8.3'
PYTHON_VERSION: '3.11'

jobs:
test-random-search-gpu:
if: github.event.pull_request.draft == false
runs-on:
- self-hosted
- Linux
- ubuntu
- gpu
steps:
- uses: actions/checkout@v4
- uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
virtualenvs-create: false
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Run Random Search tests on GPU
run: |
make env.conda
source "${HOME}/miniconda3/etc/profile.d/conda.sh"
conda activate "${{ github.workspace }}"/env
make install
cd tests
poetry run pytest --verbose \
--junitxml=./test-reports/test_random_search_report.xml \
--disable-warnings \
--verbose \
--basetemp=$HOME/actions_runner_workdir/random_search \
--input_data_directory=/mnt/data/clinicadl_data_ci/data_ci \
test_random_search.py
- name: Cleaning
run: |
rm -rf $HOME/actions_runner_workdir/random_search/*
Loading