Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] alerce.query_detections running very slow #34

Open
dharmesh1007 opened this issue Mar 25, 2023 · 0 comments
Open

[Bug] alerce.query_detections running very slow #34

dharmesh1007 opened this issue Mar 25, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@dharmesh1007
Copy link

dharmesh1007 commented Mar 25, 2023

Describe the bug
I am using the API to download light curves for a sample of sources. Each source light curve is taking several minutes to download. I am downloading in json format. I've tried the query_non_detections, query_detections, and query_lightcurve, with json and pandas format. The problem is the same. Please could you help?

To Reproduce
Steps to reproduce the behavior:

Using vscode with the latest alerce client installed.

from alerce.core import Alerce
alerce = Alerce()
try:
with open(f'{folderpath}/{oid}.json', 'w') as outfile:
json.dump(alerce.query_detections(oid, format="json"), outfile)
except Exception as e:
os.remove(f'{folderpath}/{oid}.json')
print(e)

Expected behavior
Each light curve should download within a second or two.

Environment (please complete the following information):

  • OS: macOS Ventura
  • Browser: N/A
  • Version: 13.2.1
  • Python version: 3.9.16
  • If possible the output of pip freeze to check the python environment.
    alerce==1.2.0
    anyio @ file:///opt/concourse/worker/volumes/live/485b0f52-1188-482a-6285-65a36c8fa8a6/volume/anyio_1644481714856/work/dist
    appnope @ file:///home/conda/feedstock_root/build_artifacts/appnope_1649077682618/work
    argon2-cffi @ file:///opt/conda/conda-bld/argon2-cffi_1645000214183/work
    argon2-cffi-bindings @ file:///opt/concourse/worker/volumes/live/42cf1b28-e71f-45ed-47b2-50f828088636/volume/argon2-cffi-bindings_1644569709119/work
    astropy @ file:///Users/runner/miniforge3/conda-bld/astropy_1673376509203/work
    astroquery==0.4.6
    asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1670263926556/work
    attrs @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_33k1uces4n/croot/attrs_1668696162258/work
    backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work
    backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work
    beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1675252249248/work
    bleach @ file:///opt/conda/conda-bld/bleach_1641577558959/work
    Bottleneck @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_29949159-f86f-474b-bc1f-aaa1e0e222b4ofusifik/croots/recipe/bottleneck_1657175564045/work
    brotlipy @ file:///Users/runner/miniforge3/conda-bld/brotlipy_1666764741656/work
    certifi @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_477u68wvzm/croot/certifi_1671487773341/work/certifi
    cffi @ file:///Users/runner/miniforge3/conda-bld/cffi_1671179477998/work
    charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1661170624537/work
    comm @ file:///home/conda/feedstock_root/build_artifacts/comm_1670575068857/work
    cryptography @ file:///Users/runner/miniforge3/conda-bld/cryptography_1637687153321/work
    cycler @ file:///tmp/build/80754af9/cycler_1637851556182/work
    debugpy @ file:///Users/runner/miniforge3/conda-bld/debugpy_1674522483834/work
    decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
    defusedxml @ file:///tmp/build/80754af9/defusedxml_1615228127516/work
    entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work
    exceptiongroup==1.1.0
    executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1667317341051/work
    fastjsonschema @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_b5c1gee32t/croots/recipe/python-fastjsonschema_1661368622875/work
    feets==0.4
    fonttools==4.25.0
    html5lib @ file:///home/conda/feedstock_root/build_artifacts/html5lib_1592930327044/work
    idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1663625384323/work
    imbalanced-learn @ file:///home/conda/feedstock_root/build_artifacts/imbalanced-learn_1672235740107/work
    importlib-metadata @ file:///home/conda/feedstock_root/build_artifacts/importlib-metadata_1672612343532/work
    iniconfig==2.0.0
    ipykernel @ file:///Users/runner/miniforge3/conda-bld/ipykernel_1673894854828/work
    ipython @ file:///Users/runner/miniforge3/conda-bld/ipython_1672758654845/work
    ipython-genutils @ file:///tmp/build/80754af9/ipython_genutils_1606773439826/work
    jaraco.classes @ file:///home/conda/feedstock_root/build_artifacts/jaraco.classes_1667024629799/work
    jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1669134318875/work
    Jinja2 @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_6adj7x0ejx/croot/jinja2_1666908137966/work
    joblib @ file:///home/conda/feedstock_root/build_artifacts/joblib_1663332044897/work
    jsonschema @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_d832da7jx3/croots/recipe/jsonschema_1663375475386/work
    jupyter-server @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_031akrjssy/croot/jupyter_server_1671707631142/work
    jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1673615989977/work
    jupyter_core @ file:///Users/runner/miniforge3/conda-bld/jupyter_core_1674530039210/work
    jupyterlab-pygments @ file:///tmp/build/80754af9/jupyterlab_pygments_1601490720602/work
    kaleido==0.2.1
    keyring @ file:///Users/runner/miniforge3/conda-bld/keyring_1671728304453/work
    kiwisolver @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_ca2945cc-8f2f-407b-8e24-f05d29ee2f4fvto5sdlk/croots/recipe/kiwisolver_1653292053344/work
    lasair==0.0.5
    lxml @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_1902c961-4bd2-4871-a3c5-70b7317a6521kpj7nz2o/croots/recipe/lxml_1657545138937/work
    MarkupSafe @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_d4a9444f-bd4c-4043-b47d-cede33979b0fve7bm42r/croots/recipe/markupsafe_1654597878200/work
    matplotlib @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_croot-hkd3cnd2/matplotlib-suite_1647506470899/work
    matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
    mistune @ file:///opt/concourse/worker/volumes/live/4217afd5-dad1-438d-6f79-e4992ccda0e5/volume/mistune_1607364880245/work
    mkl-fft==1.3.1
    mkl-random @ file:///opt/concourse/worker/volumes/live/0cda23d8-7460-44b2-7e5d-3c76a8a0ca7e/volume/mkl_random_1626186083266/work
    mkl-service==2.4.0
    mock==5.0.1
    more-itertools @ file:///home/conda/feedstock_root/build_artifacts/more-itertools_1677514956219/work
    munkres==1.1.4
    nbclassic @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_7c3czojxw1/croot/nbclassic_1676902906096/work
    nbclient @ file:///opt/concourse/worker/volumes/live/fcea0efc-2a08-48fd-5c55-85ef78e0ea28/volume/nbclient_1650308406463/work
    nbconvert @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_8fyzuglni_/croot/nbconvert_1668450649428/work
    nbformat @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_2daun1fill/croot/nbformat_1670352339504/work
    nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1664684991461/work
    notebook @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_0cdyriuhi_/croot/notebook_1668179888986/work
    notebook_shim @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_e9s6zsmlb7/croot/notebook-shim_1668160584892/work
    numexpr @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_cef3ah6r8w/croot/numexpr_1668713880672/work
    numpy @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_e3f85f54-0572-4c0e-b190-4bc4766fc3fenewxarhq/croots/recipe/numpy_and_numpy_base_1652801682879/work
    packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1673482170163/work
    pandas==1.5.2
    pandocfilters @ file:///opt/conda/conda-bld/pandocfilters_1643405455980/work
    parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
    patsy @ file:///home/conda/feedstock_root/build_artifacts/patsy_1665356157073/work
    pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1667297516076/work
    pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
    Pillow==9.0.1
    platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1672264874562/work
    plotly @ file:///opt/conda/envs/env/conda-bld/plotly_1674482700049/work/packages/python/plotly/dist/plotly-5.13.0.tar.gz
    pluggy==1.0.0
    prometheus-client @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_19kjbndib7/croots/recipe/prometheus_client_1659455105394/work
    prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1670414775770/work
    psutil @ file:///Users/runner/miniforge3/conda-bld/psutil_1667886143121/work
    ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
    pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work
    pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
    pyerfa @ file:///Users/runner/miniforge3/conda-bld/pyerfa_1666820916637/work
    Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1672682006896/work
    pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1608055815057/work
    pyparsing @ file:///tmp/build/80754af9/pyparsing_1635766073266/work
    pyrsistent @ file:///opt/concourse/worker/volumes/live/76cffa60-bd33-4155-4e83-ea03c38b1294/volume/pyrsistent_1636111020441/work
    PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
    pytest==7.2.2
    python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work
    pytz @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_ddzpsmm2_f/croot/pytz_1671697430473/work
    pyvo @ file:///home/conda/feedstock_root/build_artifacts/pyvo_1664228312964/work
    PyYAML @ file:///Users/runner/miniforge3/conda-bld/pyyaml_1666772473931/work
    pyzmq @ file:///Users/runner/miniforge3/conda-bld/pyzmq_1673612734897/work
    requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1673863902341/work
    scikit-learn @ file:///Users/runner/miniforge3/conda-bld/scikit-learn_1670523693646/work
    scipy @ file:///opt/concourse/worker/volumes/live/9284487f-601d-4bc3-5556-535c4949d341/volume/scipy_1641557769615/work
    seaborn @ file:///tmp/build/80754af9/seaborn_1629307859561/work
    Send2Trash @ file:///tmp/build/80754af9/send2trash_1632406701022/work
    six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
    sniffio @ file:///opt/concourse/worker/volumes/live/38ca9e9e-09d1-4d43-5a0f-b546422e7807/volume/sniffio_1614030472707/work
    soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1658207591808/work
    stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1669632077133/work
    statsmodels @ file:///Users/runner/miniforge3/conda-bld/statsmodels_1667586084161/work
    tenacity @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_15d3bbab-3059-4048-adcb-986fb2669dd5nujdx5qd/croots/recipe/tenacity_1657899116644/work
    terminado @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_18_p3gbeio/croot/terminado_1671751835656/work
    threadpoolctl @ file:///home/conda/feedstock_root/build_artifacts/threadpoolctl_1643647933166/work
    tinycss2 @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_56dshjmms6/croot/tinycss2_1668168824483/work
    tomli==2.0.1
    tornado @ file:///Users/runner/miniforge3/conda-bld/tornado_1666788801701/work
    traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1673359992537/work
    typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1665144421445/work
    urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1673452138552/work
    wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1673864653149/work
    webencodings==0.5.1
    websocket-client @ file:///opt/concourse/worker/volumes/live/5baed9cd-40fb-4fbe-6721-9568cdd0f2d7/volume/websocket-client_1614804245073/work
    xgboost==1.7.1
    zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1677313463193/work

Additional context
Add any other context about the problem here.

@dharmesh1007 dharmesh1007 added the bug Something isn't working label Mar 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant