Pyinstaller doesn't seem to work with asdf

[jeffreypaul15]

asdf doesn't seem to work with PyInstaller,

import os
import asdf

test_file =  "hgc_100.asdf"
with asdf.open(test_file) as input_asdf:
    print(input_asdf)

On running the code above with pyistaller

  File "pytesting.py", line 5, in <module>
    with asdf.open(test_file) as input_asdf:
  File "asdf/asdf.py", line 1723, in open_asdf
  File "asdf/asdf.py", line 933, in _open_impl
  File "asdf/asdf.py", line 877, in _open_asdf
  File "asdf/schema.py", line 689, in fill_defaults
  File "asdf/schema.py", line 658, in validate
  File "asdf/schema.py", line 563, in get_validator
  File "asdf/schema.py", line 263, in _create_validator
  File "asdf/schema.py", line 464, in _load_schema_cached
  File "asdf/schema.py", line 375, in load_schema
  File "asdf/schema.py", line 348, in _load_schema
  File "asdf/generic_io.py", line 980, in get_file
  File "asdf/generic_io.py", line 878, in _http_to_temp
  File "urllib/request.py", line 214, in urlopen
  File "urllib/request.py", line 523, in open
  File "urllib/request.py", line 632, in http_response
  File "urllib/request.py", line 555, in error
  File "urllib/request.py", line 494, in _call_chain
  File "urllib/request.py", line 747, in http_error_302
  File "urllib/request.py", line 523, in open
  File "urllib/request.py", line 632, in http_response
  File "urllib/request.py", line 555, in error
  File "urllib/request.py", line 494, in _call_chain
  File "urllib/request.py", line 747, in http_error_302
  File "urllib/request.py", line 523, in open
  File "urllib/request.py", line 632, in http_response
  File "urllib/request.py", line 561, in error
  File "urllib/request.py", line 494, in _call_chain
  File "urllib/request.py", line 641, in http_error_default
urllib.error.HTTPError: HTTP Error 404: Not Found

Regardless of adding a basic hook with collect_submodules and collect_data_files, the error persists.

[eslavich]

I'm not familiar with pyinstaller but I can make a couple of guesses as to why it might struggle with asdf. For one, the asdf package includes several .yaml files that it needs in order to read and validate ASDF files. The path to those files is computed relative to the filesystem location of the asdf.schema module, so if pyinstaller fails to bundle the .yaml files or changes the directory structure of the package, asdf is not going to be able to find them.

Also, depending on the content of your ASDF file, you may need one or more extension packages that provide additional .yaml files and code to convert ASDF tree objects to Python types. These extensions are discovered by asdf using the entry points mechanism and are never explicitly imported, so pyinstaller may not know about them.

[bjmcculloch]

I had some trouble with this, and found a workaround. I took the "kill it with fire" approach, and everything appears to work fine.

$ pyinstaller --clean -y --collect-all asdf --collect-all asdf_standard --collect-all asdf_transform_schemas --collect-all packaging --collect-all pkg_resources --collect-all astropy --collect-all lz4 --recursive-copy-metadata asdf --recursive-copy-metadata astropy main.py