FileExistError after updating from GitHub

[chyan26]

I got error when updating code from GitHub. I guess this can be fixed by overwriting.

Downloading METIS: 100%|█████████████████████████████████████████| 33.8M/33.8M [00:01<00:00, 28.3MB/s]
Extracting  METIS:  43%|█████████████████▌                       | 67/156 [00:00<00:00, 98.01it/s]
Traceback (most recent call last):
  File "/home/chyan/METIS_Simulations/ESO/./run_recipes.py", line 14, in <module>
    from raw_script import simulate
  File "/home/chyan/METIS_Simulations/ESO/raw_script.py", line 21, in <module>
    from simulationDefinitions import *
  File "/home/chyan/METIS_Simulations/ESO/simulationDefinitions.py", line 9, in <module>
    sim.download_packages("METIS", release="2024-05-14")
  File "/home/chyan/.local/lib/python3.12/site-packages/scopesim/server/database.py", line 413, in download_packages
    pkg_path = _download_single_package(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chyan/.local/lib/python3.12/site-packages/scopesim/server/database.py", line 333, in _download_single_package
    handle_unzipping(save_path, save_dir, pkg_name, padlen)
  File "/home/chyan/.local/lib/python3.12/site-packages/scopesim/server/download_utils.py", line 88, in handle_unzipping
    zip_ref.extract(file, save_dir)
  File "/usr/lib64/python3.12/zipfile/__init__.py", line 1716, in extract
    return self._extract_member(member, path, pwd)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/zipfile/__init__.py", line 1786, in _extract_member
    os.mkdir(targetpath)
FileExistsError: [Errno 17] File exists: 'inst_pkgs/METIS/docs/example_notebooks/inst_pkgs'

[hugobuddel]

Yeah, for now just manually delete the inst_pkgs directory. Or run run_recipes.py from a fresh directory.

Or you can comment out the sim.download_packages() line in

METIS_Simulations/ESO/simulationDefinitions.py

Line 9 in 2be93ca

sim.download_packages("METIS", release="2024-05-14")

, as instructed in

METIS_Simulations/ESO/README.md

Line 112 in 2be93ca

The first time you run this, the necessary instrument packages for ScopeSim will be downloaded; comment out the

. However, in that case you should manually ensure you are using the IRDB package version that you intent to use.

In fact, we do not want to hardcode the 2024-05-14 release anymore. We should remove the download line entirely from the code and use a git clone of the irdb internally within the team.

There are many levels to this problem. In fact, files are overwritten normally ( What triggers it is that sim.download_packages() cannot overwrite an IRDB package that is created by @teutoburg with a package that is created by me... That is because https://github.com/AstarVienna/irdb/blob/dev_master/METIS/docs/example_notebooks/inst_pkgs is a symlink. When I create the package it is zipped as a symlink, and when @teutoburg creates it, it is zipped as a regular file.

import scopesim as sim
sim.download_packages("METIS", release="2023-10-19") # Created by Fabian
sim.download_packages("METIS", release="2024-02-29") # Works, package also created by Fabian
sim.download_packages("METIS", release="2024-05-14") # Fails, package created by me

You will get the opposite problem if you switch the last two lines.

I think we should ultimately solve this by making the irdb normal pip packages.