Automatically set PATH

[bschilder]

Command

I installed gcaer within a Linux-based Bioconductor-distributed Docker container and then ran:

library(gcaer)
install_gcae()

Output

Collecting setuptools==47.1.1
  Downloading setuptools-47.1.1-py3-none-any.whl (583 kB)
Installing collected packages: setuptools
  WARNING: The scripts easy_install and easy_install-3.8 are installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed setuptools-47.1.1
Collecting tensorflow==2.2.0
  Downloading tensorflow-2.2.0-cp38-cp38-manylinux2010_x86_64.whl (516.3 MB)
Collecting opt-einsum>=2.3.2
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting gast==0.3.3
  Downloading gast-0.3.3-py2.py3-none-any.whl (9.7 kB)
Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow==2.2.0) (1.22.2)
Collecting tensorboard<2.3.0,>=2.2.0
  Downloading tensorboard-2.2.2-py3-none-any.whl (3.0 MB)
Requirement already satisfied: grpcio>=1.8.6 in /home/rstudio/.local/lib/python3.8/site-packages (from tensorflow==2.2.0) (1.46.3)
Collecting scipy==1.4.1; python_version >= "3"
  Downloading scipy-1.4.1-cp38-cp38-manylinux1_x86_64.whl (26.0 MB)
Collecting protobuf>=3.8.0
  Downloading protobuf-4.21.1-cp37-abi3-manylinux2014_x86_64.whl (407 kB)
Collecting google-pasta>=0.1.8
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting absl-py>=0.7.0
  Downloading absl_py-1.1.0-py3-none-any.whl (123 kB)
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow==2.2.0) (1.16.0)
Collecting keras-preprocessing>=1.1.0
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting tensorflow-estimator<2.3.0,>=2.2.0
  Downloading tensorflow_estimator-2.2.0-py2.py3-none-any.whl (454 kB)
Collecting h5py<2.11.0,>=2.10.0
  Downloading h5py-2.10.0-cp38-cp38-manylinux1_x86_64.whl (2.9 MB)
Collecting astunparse==1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting termcolor>=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting wrapt>=1.11.1
  Downloading wrapt-1.14.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Requirement already satisfied: wheel>=0.26; python_version >= "3" in /usr/lib/python3/dist-packages (from tensorflow==2.2.0) (0.34.2)
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting requests<3,>=2.21.0
  Downloading requests-2.28.0-py3-none-any.whl (62 kB)
Requirement already satisfied: setuptools>=41.0.0 in /home/rstudio/.local/lib/python3.8/site-packages (from tensorboard<2.3.0,>=2.2.0->tensorflow==2.2.0) (47.1.1)
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.1.2-py3-none-any.whl (224 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.7-py3-none-any.whl (97 kB)
Collecting rsa<5,>=3.1.4; python_version >= "3.6"
  Downloading rsa-4.8-py3-none-any.whl (39 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2022.6.15-py3-none-any.whl (160 kB)
Collecting importlib-metadata>=4.4; python_version < "3.10"
  Downloading importlib_metadata-4.11.4-py3-none-any.whl (18 kB)
Collecting pyasn1>=0.1.3
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting zipp>=0.5
  Downloading zipp-3.8.0-py3-none-any.whl (5.4 kB)
Building wheels for collected packages: termcolor
  Building wheel for termcolor (setup.py): started
  Building wheel for termcolor (setup.py): finished with status 'done'
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4830 sha256=78f2be1710b6c1dd34e1cba776f60a179faf36070f16a6518f5daa5f52a42bca
  Stored in directory: /home/rstudio/.cache/pip/wheels/a0/16/9c/5473df82468f958445479c59e784896fa24f4a5fc024b0f501
Successfully built termcolor
Installing collected packages: opt-einsum, gast, pyasn1, rsa, pyasn1-modules, cachetools, google-auth, urllib3, idna, charset-normalizer, certifi, requests, oauthlib, requests-oauthlib, google-auth-oauthlib, absl-py, werkzeug, protobuf, tensorboard-plugin-wit, zipp, importlib-metadata, markdown, tensorboard, scipy, google-pasta, keras-preprocessing, tensorflow-estimator, h5py, astunparse, termcolor, wrapt, tensorflow
  WARNING: The scripts pyrsa-decrypt, pyrsa-encrypt, pyrsa-keygen, pyrsa-priv2pub, pyrsa-sign and pyrsa-verify are installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script normalizer is installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script google-oauthlib-tool is installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script markdown_py is installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script tensorboard is installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts estimator_ckpt_converter, saved_model_cli, tensorboard, tf_upgrade_v2, tflite_convert, toco and toco_from_protos are installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed absl-py-1.1.0 astunparse-1.6.3 cachetools-4.2.4 certifi-2022.6.15 charset-normalizer-2.0.12 gast-0.3.3 google-auth-1.35.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 h5py-2.10.0 idna-3.3 importlib-metadata-4.11.4 keras-preprocessing-1.1.2 markdown-3.3.7 oauthlib-3.2.0 opt-einsum-3.3.0 protobuf-4.21.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.28.0 requests-oauthlib-1.3.1 rsa-4.8 scipy-1.4.1 tensorboard-2.2.2 tensorboard-plugin-wit-1.8.1 tensorflow-2.2.0 tensorflow-estimator-2.2.0 termcolor-1.1.0 urllib3-1.26.9 werkzeug-2.1.2 wrapt-1.14.1 zipp-3.8.0
Collecting numpy==1.18.4
  Downloading numpy-1.18.4-cp38-cp38-manylinux1_x86_64.whl (20.7 MB)
ERROR: pandas 1.4.1 has requirement numpy>=1.18.5; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you'll have numpy 1.18.4 which is incompatible.
Installing collected packages: numpy
  WARNING: The scripts f2py, f2py3 and f2py3.8 are installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-1.18.4
Collecting matplotlib==3.2.1
  Downloading matplotlib-3.2.1-cp38-cp38-manylinux1_x86_64.whl (12.4 MB)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib==3.2.1) (2.8.2)
Requirement already satisfied: numpy>=1.11 in /home/rstudio/.local/lib/python3.8/site-packages (from matplotlib==3.2.1) (1.18.4)
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.4.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.2 MB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.1->matplotlib==3.2.1) (1.16.0)
Installing collected packages: cycler, kiwisolver, pyparsing, matplotlib
Successfully installed cycler-0.11.0 kiwisolver-1.4.3 matplotlib-3.2.1 pyparsing-3.0.9
Collecting seaborn
  Downloading seaborn-0.11.2-py3-none-any.whl (292 kB)
Requirement already satisfied: pandas>=0.23 in /usr/local/lib/python3.8/dist-packages (from seaborn) (1.4.1)
Requirement already satisfied: numpy>=1.15 in /home/rstudio/.local/lib/python3.8/site-packages (from seaborn) (1.18.4)
Requirement already satisfied: matplotlib>=2.2 in /home/rstudio/.local/lib/python3.8/site-packages (from seaborn) (3.2.1)
Requirement already satisfied: scipy>=1.0 in /home/rstudio/.local/lib/python3.8/site-packages (from seaborn) (1.4.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=0.23->seaborn) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=0.23->seaborn) (2021.3)
Requirement already satisfied: cycler>=0.10 in /home/rstudio/.local/lib/python3.8/site-packages (from matplotlib>=2.2->seaborn) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/rstudio/.local/lib/python3.8/site-packages (from matplotlib>=2.2->seaborn) (3.0.9)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/rstudio/.local/lib/python3.8/site-packages (from matplotlib>=2.2->seaborn) (1.4.3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.8.1->pandas>=0.23->seaborn) (1.16.0)
Installing collected packages: seaborn
Successfully installed seaborn-0.11.2
Collecting numba
  Downloading numba-0.55.2-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.4 MB)
Requirement already satisfied: setuptools in /home/rstudio/.local/lib/python3.8/site-packages (from numba) (47.1.1)
Collecting llvmlite<0.39,>=0.38.0rc1
  Downloading llvmlite-0.38.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
Requirement already satisfied: numpy<1.23,>=1.18 in /home/rstudio/.local/lib/python3.8/site-packages (from numba) (1.18.4)
Installing collected packages: llvmlite, numba
Successfully installed llvmlite-0.38.1 numba-0.55.2
Collecting pandas==1.0.4
  Downloading pandas-1.0.4-cp38-cp38-manylinux1_x86_64.whl (10.0 MB)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.8/dist-packages (from pandas==1.0.4) (2021.3)
Requirement already satisfied: numpy>=1.13.3 in /home/rstudio/.local/lib/python3.8/site-packages (from pandas==1.0.4) (1.18.4)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.8/dist-packages (from pandas==1.0.4) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.6.1->pandas==1.0.4) (1.16.0)
Installing collected packages: pandas
Successfully installed pandas-1.0.4
Collecting pandas-plink==2.0.4
  Downloading pandas_plink-2.0.4-cp38-cp38-manylinux2010_x86_64.whl (88 kB)
Requirement already satisfied: pandas>=0.25.3 in /home/rstudio/.local/lib/python3.8/site-packages (from pandas-plink==2.0.4) (1.0.4)
Collecting tqdm>=4.36.1
  Downloading tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
Collecting pytest>=5.2.2
  Downloading pytest-7.1.2-py3-none-any.whl (297 kB)
Requirement already satisfied: numpy>=1.17.2 in /home/rstudio/.local/lib/python3.8/site-packages (from pandas-plink==2.0.4) (1.18.4)
Collecting dask[array,dataframe]>=2.6.0
  Downloading dask-2022.6.0-py3-none-any.whl (1.1 MB)
Collecting zstandard>=0.12.0
  Downloading zstandard-0.18.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Collecting cffi>=1.13.1
  Downloading cffi-1.15.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (446 kB)
Collecting xarray>=0.13.0
  Downloading xarray-2022.3.0-py3-none-any.whl (870 kB)
Collecting Deprecated>=1.2.6
  Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.8/dist-packages (from pandas>=0.25.3->pandas-plink==2.0.4) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.8/dist-packages (from pandas>=0.25.3->pandas-plink==2.0.4) (2021.3)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
Collecting py>=1.8.2
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting attrs>=19.2.0
  Downloading attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting tomli>=1.0.0
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: pyyaml>=5.3.1 in /usr/local/lib/python3.8/dist-packages (from dask[array,dataframe]>=2.6.0->pandas-plink==2.0.4) (6.0)
Collecting toolz>=0.8.2
  Downloading toolz-0.11.2-py3-none-any.whl (55 kB)
Collecting partd>=0.3.10
  Downloading partd-1.2.0-py3-none-any.whl (19 kB)
Collecting fsspec>=0.6.0
  Downloading fsspec-2022.5.0-py3-none-any.whl (140 kB)
Collecting cloudpickle>=1.1.1
  Downloading cloudpickle-2.1.0-py3-none-any.whl (25 kB)
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
Requirement already satisfied: wrapt<2,>=1.10 in /home/rstudio/.local/lib/python3.8/site-packages (from Deprecated>=1.2.6->pandas-plink==2.0.4) (1.14.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.6.1->pandas>=0.25.3->pandas-plink==2.0.4) (1.16.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/rstudio/.local/lib/python3.8/site-packages (from packaging->pytest>=5.2.2->pandas-plink==2.0.4) (3.0.9)
Collecting locket
  Downloading locket-1.0.0-py2.py3-none-any.whl (4.4 kB)
ERROR: xarray 2022.3.0 has requirement pandas>=1.1, but you'll have pandas 1.0.4 which is incompatible.
Installing collected packages: tqdm, iniconfig, packaging, py, attrs, tomli, pluggy, pytest, toolz, locket, partd, fsspec, cloudpickle, dask, zstandard, pycparser, cffi, xarray, Deprecated, pandas-plink
  WARNING: The script tqdm is installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts py.test and pytest are installed in '/home/rstudio/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Deprecated-1.2.13 attrs-21.4.0 cffi-1.15.0 cloudpickle-2.1.0 dask-2022.6.0 fsspec-2022.5.0 iniconfig-1.1.1 locket-1.0.0 packaging-21.3 pandas-plink-2.0.4 partd-1.2.0 pluggy-1.0.0 py-1.11.0 pycparser-2.21 pytest-7.1.2 tomli-2.0.1 toolz-0.11.2 tqdm-4.64.0 xarray-2022.3.0 zstandard-0.18.0

Error

When I go to run subsequent functions, they have trouble finding the scripts:

 get_gcae_help_text()

 [1] "Traceback (most recent call last):"                                                                                                                            
 [2] "  File \"/home/rstudio/.local/share/GenoCAE/run_gcae.py\", line 31, in <module>"                                                   
 [3] "    import tensorflow as tf"                                                                                                       
 [4] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/__init__.py\", line 41, in <module>"                          
 [5] "    from tensorflow.python.tools import module_util as _module_util"                                                               
 [6] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/python/__init__.py\", line 53, in <module>"                   
 [7] "    from tensorflow.core.framework.graph_pb2 import *"                                                                             
 [8] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/core/framework/graph_pb2.py\", line 16, in <module>"          
 [9] "    from tensorflow.core.framework import function_pb2 as tensorflow_dot_core_dot_framework_dot_function__pb2"                     
[10] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py\", line 16, in <module>"       
[11] "    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2"                
[12] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py\", line 16, in <module>"     
[13] "    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2"                         
[14] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py\", line 16, in <module>"         
[15] "    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2"      
[16] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py\", line 16, in <module>"
[17] "    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2"            
[18] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py\", line 36, in <module>"   
[19] "    _descriptor.FieldDescriptor("                                                                                                  
[20] "  File \"/home/rstudio/.local/lib/python3.8/site-packages/google/protobuf/descriptor.py\", line 560, in __new__"                   
[21] "    _message.Message._CheckCalledFromGeneratedFile()"                                                                              
[22] "TypeError: Descriptors cannot not be created directly."                                                                            
[23] "If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0."          
[24] "If you cannot immediately regenerate your protos, some other possible workarounds are:"                                            
[25] " 1. Downgrade the protobuf package to 3.20.x or lower."                                                                            
[26] " 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower)."            
[27] ""                                                                                                                                  
[28] "More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates"                              
attr(,"status")
[1] 1
Warning message:
In system2(command = run_args[1], args = run_args[-1], stdout = TRUE,  :
  running command ''python3' ~/.local/share/GenoCAE/run_gcae.py --help 2>&1' had status 1

Attempted fixes

I tried exporting the directory to PATH, but rerunning get_gcae_help_text after doing this didn't seem to help (same errors as before).

Sys.setenv("PATH"=paste(Sys.getenv("PATH"),"/home/rstudio/.local/bin",sep = ":"))

I can't be sure, but it doesn't seem like the python dependencies are being installed via conda. Would this perhaps help to alleviate some of these issues and reduce potential conflicts with existing installations? basilisk is quite useful for managing these environments in a CRAN/Bioc-friendly way.

I've also made the package echoconda for extending basilisk/reticulate to find exact paths to conda-installed executables, which you might find helpful in some cases. It's currently only installable via remotes but I do plan on submitting it to CRAN at some point.

Session info

R Under development (unstable) (2022-02-25 r81808)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.3 LTS

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] gcaer_0.6.6

loaded via a namespace (and not attached):
  [1] rappdirs_0.3.3                rtracklayer_1.56.0            semver_0.2.0                  R.methodsS3_1.8.2            
  [5] coda_0.19-4                   tidyr_1.2.0                   ggplot2_3.3.6                 clusterGeneration_1.3.7      
  [9] bit64_4.0.5                   knitr_1.39                    DelayedArray_0.22.0           R.utils_2.11.0               
 [13] data.table_1.14.2             KEGGREST_1.36.2               RCurl_1.98-1.7                GEOquery_2.64.2              
 [17] generics_0.1.2                BiocGenerics_0.42.0           GenomicFeatures_1.48.3        callr_3.7.0                  
 [21] usethis_2.1.6                 RSQLite_2.2.14                combinat_0.0-8                bit_4.0.4                    
 [25] tzdb_0.3.0                    webshot_0.5.3                 xml2_1.3.3                    httpuv_1.6.5                 
 [29] SummarizedExperiment_1.26.1   assertthat_0.2.1              orthogene_1.2.0               viridis_0.6.2                
 [33] gargle_1.2.0                  xfun_0.31                     hms_1.1.1                     babelgene_22.3               
 [37] evaluate_0.15                 promises_1.2.0.1              TSP_1.2-0                     fansi_1.0.3                  
 [41] restfulr_0.0.14               progress_1.2.2                dendextend_1.15.2             dbplyr_2.2.0                 
 [45] readxl_1.4.0                  igraph_1.3.2                  DBI_1.1.2                     htmlwidgets_1.5.4            
 [49] stats4_4.2.0                  purrr_0.3.4                   ellipsis_0.3.2                dplyr_1.0.9                  
 [53] ggpubr_0.4.0                  backports_1.4.1               gprofiler2_0.2.1              biomaRt_2.52.0               
 [57] MatrixGenerics_1.8.0          MungeSumstats_1.5.1           vctrs_0.4.1                   SingleCellExperiment_1.18.0  
 [61] Biobase_2.56.0                remotes_2.4.2                 here_1.0.1                    abind_1.4-5                  
 [65] cachem_1.0.6                  withr_2.5.0                   sys_3.4                       BSgenome_1.64.0              
 [69] vroom_1.5.7                   GenomicAlignments_1.32.0      treeio_1.20.0                 prettyunits_1.1.1            
 [73] mnormt_2.1.0                  phytools_1.0-3                ExperimentHub_2.4.0           ape_5.6-2                    
 [77] lazyeval_0.2.2                crayon_1.5.1                  pkgconfig_2.0.3               GenomeInfoDb_1.32.2          
 [81] nlme_3.1-155                  pkgload_1.2.4                 seriation_1.3.5               devtools_2.4.3               
 [85] ewceData_1.4.0                rlang_1.0.2                   lifecycle_1.0.1               registry_0.5-1               
 [89] filelock_1.0.2                BiocFileCache_2.4.0           AnnotationHub_3.4.0           cellranger_1.1.0             
 [93] rprojroot_2.0.3               matrixStats_0.62.0            Matrix_1.4-0                  aplot_0.1.6                  
 [97] phangorn_2.8.1                carData_3.0-5                 boot_1.3-28                   processx_3.6.0               
[101] png_0.1-7                     viridisLite_0.4.0             rjson_0.2.21                  bitops_1.0-7                 
[105] R.oo_1.25.0                   Biostrings_2.64.0             blob_1.2.3                    stringr_1.4.0                
[109] plinkr_0.20.3                 readr_2.1.2                   rstatix_0.7.0                 MAGMA.Celltyping_2.0.4       
[113] gridGraphics_0.5-1            S4Vectors_0.34.0              ggsignif_0.6.3                scales_1.2.0                 
[117] memoise_2.0.1                 magrittr_2.0.3                plyr_1.8.7                    zlibbioc_1.42.0              
[121] compiler_4.2.0                BiocIO_1.6.0                  RColorBrewer_1.1-3            plotrix_3.8-2                
[125] lme4_1.1-29                   homologene_1.4.68.19.3.27     Rsamtools_2.12.0              cli_3.3.0                    
[129] XVector_0.36.0                patchwork_1.1.1               ps_1.7.0                      MASS_7.3-55                  
[133] tidyselect_1.1.2              stringi_1.7.6                 yaml_2.3.5                    askpass_1.1                  
[137] credentials_1.3.2             grid_4.2.0                    VariantAnnotation_1.42.1      fastmatch_1.1-3              
[141] tools_4.2.0                   parallel_4.2.0                rstudioapi_0.13               RNOmni_1.0.0                 
[145] foreach_1.5.2                 gridExtra_2.3                 ormr_0.6.2.2                  scatterplot3d_0.3-41         
[149] HGNChelper_0.8.1              digest_0.6.29                 BiocManager_1.30.18           shiny_1.7.1                  
[153] gert_1.6.0                    quadprog_1.5-8                Rcpp_1.0.8.3                  GenomicRanges_1.48.0         
[157] car_3.0-13                    broom_0.8.0                   BiocVersion_3.15.2            later_1.3.0                  
[161] httr_1.4.3                    ggdendro_0.1.23               AnnotationDbi_1.58.0          colorspace_2.0-3             
[165] brio_1.1.3                    XML_3.99-0.10                 fs_1.5.2                      reticulate_1.25              
[169] IRanges_2.30.0                splines_4.2.0                 uwot_0.1.11                   yulab.utils_0.0.4            
[173] tidytree_0.3.9                expm_0.999-6                  ggplotify_0.1.0               plotly_4.10.0                
[177] sessioninfo_1.2.2             xtable_1.8-4                  jsonlite_1.8.0                nloptr_2.0.3                 
[181] ggtree_3.4.0                  heatmaply_1.3.0               ggfun_0.0.6                   testthat_3.1.4               
[185] R6_2.5.1                      EWCE_1.5.1                    pillar_1.7.0                  htmltools_0.5.2              
[189] mime_0.12                     glue_1.6.2                    fastmap_1.1.0                 minqa_1.2.4                  
[193] BiocParallel_1.30.3           interactiveDisplayBase_1.34.0 codetools_0.2-18              maps_3.4.0                   
[197] pkgbuild_1.3.1                utf8_1.2.2                    lattice_0.20-45               tibble_3.1.7                 
[201] numDeriv_2016.8-1.1           curl_4.3.2                    waldo_0.4.0                   openssl_2.0.2                
[205] limma_3.52.1                  rmarkdown_2.14                googleAuthR_2.0.0             desc_1.4.1                   
[209] munsell_0.5.0                 GenomeInfoDbData_1.2.8        iterators_1.0.14              reshape2_1.4.4               
[213] gtable_0.3.0 

[richelbilderbeek]

Hi @bschilder, thanks for this detailed Issue!

I assume you've seen the ERRORs in the output you sent?

Collecting numpy==1.18.4
  Downloading numpy-1.18.4-cp38-cp38-manylinux1_x86_64.whl (20.7 MB)
ERROR: pandas 1.4.1 has requirement numpy>=1.18.5; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you'll have numpy 1.18.4 which is incompatible.

and

Collecting locket
  Downloading locket-1.0.0-py2.py3-none-any.whl (4.4 kB)
ERROR: xarray 2022.3.0 has requirement pandas>=1.1, but you'll have pandas 1.0.4 which is incompatible.

The problems are due to GenoCAE being pinned to many specific versions. As gcaer 'only' allows one to run GenoCAE through R, I will not touch the requirements.txt file in the main GenoCAE repo, but you could request her to use more recent versions of the Python libraries.

I guess it is those rather old Python versions that cause your errors, both in the installation and in the gcaer::get_gcae_help_text() function.

I can't be sure, but it doesn't seem like the python dependencies are being installed via conda.

I would have enjoyed using conda, but then I get the same problems as you: I cannot install the Python packages needed by GenoCAE, as these are too old. Instead, I use pip (as can be seen in the ormr package).

I would consider basilisk would it be a CRAN package. I intend gcaer to be a CRAN package one day, and CRAN policy is that a CRAN package may only depend on CRAN packages. I will reconsider when (and maybe it already is?) that requirement is lifted. Then, similar to conda, maybe basilisk cannot install the current (old) GenoCAE requirements as well. Also, I need a volunteer (you?) before adding this :-)

TLDR: GenoCAE uses old versions of Python packages and that caused your errors. gcaer does not touch these GenoCAE requirements. You could contact the GenoCAE authors and ask for newer versions. Using basilisk has many questions, but if 'yes' to all of them, sure, I add it!

[bschilder]

Thanks for the quick reply @richelbilderbeek. I actually somehow missed those error messages, but that makes sense now.

The problems are due to GenoCAE being pinned to many specific versions. As gcaer 'only' allows one to run GenoCAE through R, I will not touch the requirements.txt file in the main GenoCAE repo, but you could request her to use more recent versions of the Python libraries.

I just recently created a conda env yaml file and this seems to be working perfectly when using the original implementation of GCAE (without an R wrapper), even without using a Docker container. So it seems conda may be indeed be helpful for GCAER as well.
kausmees/GenoCAE#32

I would consider basilisk would it be a CRAN package. I intend gcaer to be a CRAN package one day, and CRAN policy is that a CRAN package may only depend on CRAN packages. I will reconsider when (and maybe it already is?) that requirement is lifted. Then, similar to conda, maybe basilisk cannot install the current (old) GenoCAE requirements as well.

I believe CRAN allows you to have dependencies from either CRAN or Bioc. So in theory basilisk should be okay to use for GCAER.

Also, I need a volunteer (you?) before adding this :-)

Happy to! Will set up a time to video chat via email.