Remove GDAL and RichDEM dependancy from tests

[vschaffn]

Resolves #671.

Description

The purpose of this PR is to eliminate dependencies on GDAL and RichDEM in the xDEM tests, thereby simplifying the installation process for developers. The scripts related to GDAL and RichDEM have been relocated to the xdem-data repository, where they generate and save the ground truth data in .tif format. These .tif files are then imported into xDEM during test execution.

Changes

• The functions related to GDAL and RichDEM have been removed from conftest.py and the test files, and transferred to xdem-data. These functions have been replaced by calls to the ground truths generated within xdem-data.

• New functions have been added to conftest.py to retrieve the ground truth data from the xdem-data repository.

• The GDAL and RichDEM packages have been removed from the configuration files.

• The Makefile has been simplified and restored for Python versions above 3.10.

• The version of the scipy package has been fixed to <1.15.0, as the new version causes issues with test_biascorr.py. A new issue has been opened to modify this when an update of scipy will be released Fix scipy<1.15.0 pin in virtual environment configuration files once issue in SciPy is resolved #677.

Note

Before merging:

• Generate GDAL and RichDEM ground truths for xDEM tests xdem-data#3 should be merged
• _TESTDATA_REPO_URL and _COMMIT_HASH in conftest.py would have to be updated to point the correct repository

[duboise-cnes]

^mease

[duboise-cnes]

I quickly checked if downloading was the best solution, and it seems fine and easy to maintain. Indeed, it might be better to download a tarball via a release. Is that what you were thinking? Also, we obviously need to update this PR once the xdem-data PR is merged and a release is created, so we can have the correct tarball/zip with a fixed version API. It doesn't seem like you're handling a version here. Maybe it's better to use a release, something like:

https://github.com/glaciohack/xdem-data/releases/download/v1.0.0/xdem-data.zip

[vschaffn]

@duboise-cnes I used the same download system as for importing the example data (Longyearbyen DEMs) here to maintain the same consistency and ease of maintenance. The example data are downloading from the main xdem-data repository which is why the aim is to merge the changes made to xdem-data so that one can also point to the main repository to download the test data. (i. e. to replace the url by https://github.com/GlacioHack/xdem-data/tarball/main with time)

[duboise-cnes]

ok I understand therefore the rationale, you know better the code of xdem, i didn't know that there was the example download code. I would say that using requests is better than urllib for maintenance but these would change the entire also for the examples.

And the download function could not be factorized with the example data, some code replication here. Not needed in my opinion.
Maybe add a global xdem function to download and avoid the duplication of code (and therefore ease maintenance and evolution of downloading if needed afterwards). @adebardo @adehecq your opinion ?
My 2cts.

[duboise-cnes]

I think that you can use requests et zipfile (or tarball as used equivalent) if a release ?

For example from a well known AI ;) not used directly just for inspiration

import os
import requests
import zipfile
from pathlib import Path

def download_reference_data(data_dir="tests/data", url="https://github.com/nom_utilisateur/xdem-data/releases/download/v1.0.0/xdem-data.zip"):
    """Télécharge et extrait les données de référence si elles ne sont pas déjà présentes."""
    data_path = Path(data_dir)
    
    # Vérifie si les données existent déjà
    if data_path.exists() and any(data_path.iterdir()):
        print("Les données de référence sont déjà disponibles.")
        return
    
    # Crée le répertoire si nécessaire
    data_path.mkdir(parents=True, exist_ok=True)
    
    # Télécharge l'archive ZIP
    print(f"Téléchargement des données de référence depuis {url}...")
    response = requests.get(url)
    response.raise_for_status()
    
    # Extraction de l'archive ZIP
    with zipfile.ZipFile(io.BytesIO(response.content)) as zip_ref:
        zip_ref.extractall(data_dir)
    
    print(f"Données de référence extraites dans {data_dir}.")

does it help ?

[duboise-cnes]

do not needed with requests ?

[duboise-cnes]

check the format with a versioned release in xdem-data. if tar ok, otherwise zip equivalent.

[duboise-cnes]

why <1.15.0 ? not great to keep that. If really needed, please add an issue to correct it in another PR. This can be difficult with other packages in time. I prefer scipy >=, !=1.15.0 . it allows that next scipy release automatic upgrade and test if bugs are resolved.

[vschaffn]

#677

[duboise-cnes]

same review

[duboise-cnes]

same

[duboise-cnes]

@adehecq @rhugonnet I have made some review, please tell if other remarks to be able to merge. The richdem, gdal removal was discussed with @rhugonnet .

Have a good day